Optimizely Search and Navigation – Part 1 – Search Tips

Introduction

Search and Navigation is a cloud service provided by Optimizely to support building search functionality for both Optimizely CMS and Optimizely Commerce sites. This platform uses Elasticsearch behind the scenes to serve search, filter, facet, and indexing functionalities with the strong capabilities of Elasticsearch.

I have experience applying Search & Navigation to both CMS and Commerce projects. So today, I will share some tips for searching with you.

Which version of Optimizely CMS and Search & Navigation

The platform versions that I am using for this article are Optimizely CMS 12.27.x and EpiServer.Find 16.1.x

Difference between search and filter

Search Concept:

• The search concept in Elasticsearch refers to the process of querying the index to retrieve documents that match certain criteria.
• When you perform a search, Elasticsearch calculates a relevance score for each document based on how well it matches the query.
• Search queries can be performed using various query DSL (Domain Specific Language) constructs such as match queries, term queries, range queries, etc.
• Search results are typically sorted by relevance, ut you can also specify sorting criteria based on specific fields.

Filter Concept:

• Filters, on the other hand, are used to narrow down the set of documents returned by a query without affecting the relevance scores.
• Filters are typically used for exact matches or ranges and are applied to the documents after the initial search.
• Filters can significantly improve the performance of queries, especially for frequently used and static criteria.
• Unlike search queries, filters are not scored, so they are faster but less flexible in terms of matching documents.

In summary, while both searches and filters are used to retrieve documents from Elasticsearch, searches are used to find documents based on relevance scores calculated against the query, while filters are used to narrow down the set of documents without affecting relevance scoring.

How to apply free-text search with contains operator

We can apply the contains operator for free-text search by using wildcard as follows:

query = query.For(searchTerm, options => {
    options.Query = $"*{searchTerm}*";
    options.AllowLeadingWildcard = true;
    options.AnalyzeWildcard = true;
    options.RawQuery = searchTerm;
});

AllowLeadingWildcard: If true, the wildcard characters * and ? are allowed as the first character of the query string. Defaults to true

AnalyzeWildcard: If true, the query attempts to analyze wildcard terms in the query string. Defaults to false.

The below image is the result when I find products via free-text search - contains operator - In Name field:

How free-text search works if a search term contains multiple words

Actually, when we do a free-text search with For method by default then your search term will be analyzed to separated words and do search by separated words with OR operator by default. You can change the operator to AND by this setting:

query = query.For(searchTerm, options => {
    options.DefaultOperator = EPiServer.Find.Api.Querying.Queries.BooleanOperator.And;
})});

OR

query = query.For(searchTerm) }).WithAndAsDefaultOperator();

The below image is the result when I find products via free-text search - AND operator - In Name field:

Note: The AND operator here means that search results should contain all words in the search term without order respect.

How to apply the free-text search for the whole phrase

Sometimes you want to search contents by the exact word or phrase that you input. So in order to apply the free-text search for the whole phrase, you can wrap your phrase with this format “{words}” like this:

query = query.For($”\”{searchTerm}\””);

The below image is the result when I find products via free-text search - whole phrase matching - In Name field:

How to apply fuzzy search

Fuzzy search can be used to search for content even when the user makes typographical errors, or when you want to return all search results that match the exact keyword as well as the approximate keyword. However, you need to consider whether you really want to use fuzzy search or not because it does not perform as well as normal free-text search.

Here is a way that I used to apply fuzzy search:

First, adding an extension method to build a fuzzy query

public static ITypeSearch<TSource> FuzzySearch<TSource>(this ITypeSearch<TSource> search, string query, Expression<Func<TSource, string>> fieldSelector, double minSimilarity, int? prefixLength)
{
    return new Search<TSource, QueryStringQuery>(
        search,
        (ISearchContext context) =>
        {
            if (minSimilarity <= 0 || minSimilarity >= 1)
            {
                return;
            }

            var boolQuery = new BoolQuery();

            boolQuery.MinimumNumberShouldMatch = 1;

            boolQuery.Should.Add(context.RequestBody.Query);

            var fieldNameForSearch = search.Client.Conventions.FieldNameConvention
    .GetFieldNameForAnalyzed(fieldSelector);

            var fuzzyQuery = new FuzzyQuery(fieldNameForSearch, query)
            {
                MinSimilarity = minSimilarity,
                PrefixLength = prefixLength,
                Boost = 0.5,
            };

            boolQuery.Should.Add(fuzzyQuery);

            context.RequestBody.Query = boolQuery;
        });
}

MinSimilarity: It means the minimum similarity is acceptable to fit the result

PrefixLength: Number of beginning characters left unchanged for fuzzy matching. Defaults to 0

Boost: Factor for relevance score of fuzzy query. The default is 1. But we can change it based on our purpose to make sure the default result order based on the score as your expectation. 

Second, you can use this method for your search object as follows:

query = query.FuzzySearch(searchTerm, x => x.Name, 0.5, 0);

The below image is the result when I find products via fuzzy search - Similarity is 0.5 - In Name field:

Note: I tried to apply fuzzy search by this guide https://docs.developers.optimizely.com/digital-experience-platform/v1.1.0-search-and-navigation/docs/fuzzy-search but it does not work with me. I will spend time to dig deeply to find a reason later.

How to apply multiple search queries with AND/OR operator

If you have more than 1 query and need to define AND/OR operator for them then you can do by using BoolQuery.

Here is the way to use BoolQuery for OR:

var boolQuery = new BoolQuery();

boolQuery.MinimumNumberShouldMatch = 1;

boolQuery.Should.Add(subQuery);

Here is the way to use BoolQuery for AND:

var boolQuery = new BoolQuery();

boolQuery.Must.Add(subQuery);

Or AND for negative queries

var boolQuery = new BoolQuery();

boolQuery.MustNot.Add(subQuery);

Conclusion

In my opinion, using Search & Navigation is still a good choice when you need a search solution for your application. It uses Elasticsearch behind the scenes with a lot of available support for search, filtering, paging, faceting, and statistics, among other features. I hope that the search tips above can help you when applying Search & Navigation to your project.