How to add search to a website using flexsearch

Adding search functionality to your website can greatly enhance the user experience by making it easier for visitors to find the information they're looking for. Without search, users may have to navigate through multiple pages or rely on site navigation, which can be frustrating, especially on larger websites. Search allows users to quickly find relevant content by entering keywords or phrases.

This is especially true if you are creating documentation for a software product or api. Simply because all the information is not useful to the user, often user wants to accomplish something and is looking for specific information and wants to get there as quickly as possible there is no time for reading the entire documentation.

When it comes to adding search functionality to your website, you have several options to choose from. You can opt for commercial solutions or open-source alternatives, including local search that does not involve any server or server-based search, as well as solutions tailored for static websites or dynamic content management systems (CMS).

Using local search libraries, such as FlexSearch, involves following steps:

1. Creating Index: The first step is to create an index of the content you want to make searchable. This typically involves extracting relevant text data from your website or application, such as titles, descriptions, summaries, or full content, depending on your specific use case.
2. Adding Index to the Library: Once you have extracted the necessary text data, you need to add it to the search library. This enables the library to efficiently search through the content and retrieve relevant results based on user queries.
3. Searching the data: Third step is to search the data with user query.This typically involves creating a search function that takes user input and performs the search against the index.
4. Displaying results: Final step is to display results. To make the search functionality accessible to users, you'll need to create a user interface (UI) that allows users to enter their search queries and displays the search results. This can be done using HTML, CSS, and JavaScript, and can be integrated into your existing website design. In your website's HTML, you'll need to create a search input field and a container for displaying the results. Then, using JavaScript, you can capture user input, trigger searches, and dynamically display the search results.

Lets take an example with Flexsearch to make it more clear. FlexSearch is a lightweight and fast client-side search library written in JavaScript. It is designed to provide efficient full-text search capabilities for web applications, including static websites.

Compared to other client-side search libraries, FlexSearch offers superior performance, especially when dealing with large datasets or complex search queries. It supports advanced features such as phonetic search, relevance scoring, and pagination, making it a versatile solution for a wide range of use cases.

While the example provided focuses on FlexSearch, it's worth noting that other local libraries generally follow similar steps with minor differences in their APIs. The fundamental process of creating an index, adding data, searching, and displaying results remains consistent across these libraries.

Here's a brief example demonstrating how to use FlexSearch for local search:

    // Step 1: Example data this needs to be extracted from website in this form final result is shown here
    const documents = [
        { id: '1', title: 'Sample Title 1', description: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit.' },
        { id: '2', title: 'Sample Title 2', description: 'Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.' },
        // Add more documents as needed
    ];

    // Step 2: Adding index/document to the library
    var index = new FlexSearch.Document({
        tokenize: "forward",
        cache: 100,
        document: {
            id: 'id',
            store: [
                "title", "description"
            ],
            index: ["title", "description"]
        }
    });

    documents.forEach(item => {
        index.add({
            id: item.id,
            title: item.title,
            description: item.description
        });
    });

    // Step 3: Searching
    const query = 'sample'; // Example search query
    const results = index.search(query);

    // Step 4: Display search results
    console.log(results);

In this example, we create an index using FlexSearch and add sample documents containing titles and description. We then perform a search using a sample query and display the results. This demonstrates the basic usage of FlexSearch for implementing local search functionality.

Now let's see how to create document index for a website. Since this part lies outside the scope of Flexsearch library we skipped it in the previous Flexsearch example.

Hugo is a widely-used static site generator known for its flexibility and ease of use. If you're using Hugo, you can take advantage of its data templates and built-in functions to create a search index and integrate FlexSearch into your website. Hugo's data templates enable you to generate JSON files containing your website's content. By creating a template that extracts the required data, such as page titles and summaries, you can easily generate a JSON file for FlexSearch indexing. Here's a step-by-step guide on how to integrate flexsearch with hugo

• Creating a Data Template for Indexing: First, create a data template file (e.g., `layouts/_default/search.json`) that generates the search index based on your content. Here's an example that indexes the title and description of your posts:

     {{ $list := slice }}
     {{- $list = (where .Site.RegularPages.ByTitle "Section" .Section) }}
     {{ $currentSection := .CurrentSection }}
     {{ $jsonArray := slice }}
     {{ range $index, $element := $list -}}
     {{ if in (.RelPermalink | string) $currentSection.RelPermalink }}
     {{ if not .Params.exclude }}
  
     {{ $jsonObject := dict "id" $index "href" .RelPermalink "title" (.Title | jsonify) }}
     {{ with .Description -}}
     {{ $jsonObject = merge $jsonObject (dict "description" (. | jsonify)) }}
     {{ else -}}
     {{ $jsonObject = merge $jsonObject (dict "description" (.Summary | plainify | jsonify)) }}
     {{ end -}}
  
     {{ $jsonArray = $jsonArray | append $jsonObject }}
     {{ end }}
     {{ end }}
     {{ end -}}
     {{ $jsonArray | jsonify }}

  This template iterates over your site's regular pages, creates a dictionary with the title, description, and permalink for each page, and appends it to the `jsonArray` variable. Finally, it converts the `jsonArray` variable to JSON

• Using the Data Template in Content Section: Next, you will need to incorporate this data template into your content section to generate the index. Create a search.md file in section where you want to enable search. For instance if you want to enable it on docs than add it to the `content/docs/` folder

  +++
  layout = "search"
  outputs = ['json']
  exclude = true
  +++

• Adding FlexSearch JavaScript Library and Loading Index Data: Download the FlexSearch library and add it to your Hugo project's static folder, for instance, static/js/flexsearch.js. Then, include the library in your HTML template. Create a JavaScript file that includes the FlexSearch library and initializes the search functionality.

  Utilize Hugo's built-in asset pipeline or a custom script to include the FlexSearch JavaScript file in your website.

  <script src="/js/flexsearch.js"></script>

  Next, load the index data into FlexSearch and set up the search functionality. Here's how you can modify the code to achieve this:

        <script>
        // Initialize FlexSearch library
    var index = new FlexSearch.Document({
        tokenize: "forward",
        cache: 100,
        document: {
            id: 'id',
            store: [
                "href", "title", "description"
            ],
            index: ["title", "description"]
        }
    });
    // Load index data asynchronously
    // dataUrl is the url that contains search.json
    fetch(dataUrl)
        .then(response => response.json())
        .then(data => {
            // Loop through the data and add it to the index
            data.forEach(item => {
                index.add({
                    id: item.id,
                    href: item.href,
                    title: item.title,
                    description: item.description
                });
            });
        })
        .catch(error => {
            console.error('Error fetching data:', error);
        });
    </script>

In this code:

• We use the `fetch` API to asynchronously load the index data (`search.json`) from the server.
• Once the data is loaded, we initialize a FlexSearch and add the index data to it.

Querying and Displaying results

  <!--- //sample code for setting up user input and display result in html--->
  <input type="text" id="searchInput" placeholder="Search...">
  <ul id="searchResults"></ul>

  <script src="/js/flexsearch.js"></script>
  <script>
    // JavaScript code snippets go here
  </script>

• This is sample code to create a search input field and results container.

  // sample code for querying and displaying result
  var inputElement = document.getElementById('searchInput');
  var resultsElement = document.getElementById('searchResults');

  inputElement.addEventListener('input', function() {
      var query = inputElement.value;
      var results = index.search(query);

      // Display search results
      resultsElement.innerHTML = '';
      results.forEach(function(result) {
          var li = document.createElement('li');
          li.textContent = result.title;
          resultsElement.appendChild(li);
      });
  });

• This is sample code tp use JavaScript to capture user input, trigger searches using FlexSearch, and dynamically update the search results.

With these steps, you can integrate FlexSearch for local search functionality in your Hugo static site.

Here is the list of other search libraries for reference

Flexsearch is a great library that can be used to add local search to your website. I am planning to change the theme of Easyanalytics so my effort of implementing Flexsearch based search is likely to go waste but it was good to learn how local search is implemented. There are some more use cases like searching for ai prompts so you don't have to keep typing them again and searching data from your bookmarks before searching on internet where this kind of approach can be useful.