Loading...
Area: Optimizely B2B Commerce

Search pipelines

Recommended reading 

CategoryIndexerPipeline

CreateElasticsearchCategory

Order Pipe Description
100 CreateElasticsearchCategory Create a new ElasticsearchCategory object using the parameter object's fields and assigns it to result.ElasticsearchCategory.

GetIndexableCategories

Order Pipe Description
100 GetIndexableCategoriesQuery Creates an Entity Framework Category query using the parameter object's fields and assigns it to result.IndexableCategories.

MapElasticSearchCategoryProperty

Order Pipe Description
100 MapElasticSearchCategoryCustomProperties If the parameter.SearchBoost.IsQueryable is true and parameter.PropertyDefinition.IsCustomProperty is true sets result.Property to a new Nest TextProperty.

CategorySearchPipeline

RunCategorySearch

Order Pipe Description
100 InitializeSearchProperties Retrieves the current SiteContext, ElasticsearchIndex and ElasticsearchQueryBuilder and assigns them to the result object.
200 GetSettings Retrieves SearchGeneralSettings and FuzzySearchSettings and assigns them to the result object.
300 GetQueryFields Assigns the result.CategorySearchFields object with the fields that are queryable.
400 FormPaging Assigns result.StartRow and result.PageSize from the parameter object.
500 FormCategoryExcludeList Finds the disallowed categories for the current user (using the FormProductFilter pipeline) and caches this value. The value is then assigned to result.DisallowedCategories.
600 FormCategoryExcludeFilter If there are any disallowedCategories adds a new negated Terms query to result.AllFilters using result.DisallowedCategories on the categoryId field.
700 FormLanguageFilter Adds a new MatchQuery using the result.SiteContext.LanguageDto.LanguageCode on the languageCode field to result.AllFilters.
800 FormWebsiteFilter Adds a new MatchQuery using the result.SiteContext.WebsiteDto.Id on the websiteId field to result.AllFilters.
900 CombineFilters Creates a BooleanQuery (AND) of all filters in result.AllFilters and assigns it to result.CombinedFilter.
1000 FormMainSearchQuery

Creates a MultiMatchQuery of type CrossFields using parameter.Criteria and result.CategorySearchFields and apply boosts.

Creates a MultimatchQuery of type PhrasePrefix using parameter.Criteria and result.CategorySearchFields and apply boosts.

Adds both of these queries to result.AllQueries.

1100 FormFuzzySearchQuerry

If fuzzy search is enabled, creates a MultiMatchQuery with fuzziness set to FuzzySearchSettings.MaxEdits using parameter.Criteria and result.CategorySearchFields and apply boosts.

Adds this query to result.AllQueries.

1200

CombineQueries Creates a BooleanQuery (OR) of all queries in result.AllQueries and assigns it to result.CombinedQuery.
1300 ApplyQueryAndFilter Creates a SearchDescriptor with a FunctionScore query using result.CombinedQuery and result.CombinedFilter, with a FieldValueFactor function using the boost field and assign it to result.SearchDescriptor.
1400 ApplyPaging Sets result.SearchDescriptor's From and Size variables using result.StartRow and result.PageSize.
1500 ApplySortOrder Sets result.SearchDescriptor's sort to sort on the sortOrder field in descending order then on _score in descending order.
1600 PerformSearch Uses result.ElasticsearchIndex.Client to call Elasticsearch using result.SearchDescriptor and assigns the result to result.SearchResponse.
1700 ProcessSearchResponse If result.SearchResponse is not null and is valid, creates a new CategorySearchResult with the search result count and an IEnumerable of CategorySearchResultDto objects using result.SearchResponse.Documents then assigns this to result.CategorySearchResult.

ContentIndexerPipeline

CreateElasticsearchContent

Order Pipe Description
100 CreateElasticsearchContent

Creates a new ElasticsearchContent using the parameter object's fields and assigns it to result.ElasticsearchContent.

Fields that vary based on language, persona are populated by filtering parameter.IndexableContentItem.Fields for the field with the appropriate context.

GetIndexableContentItems

Order Pipe Description
100 GetIndexableContentItemQuery Creates an Entity Framework IndexableContentItem query using the parameter object's fields and assigns it to result.IndexableContentItems.

ContentSearchPipeline

RunContentSearch

Order Pipe Description
100 ValidatePageSize Sets result.PageSize based on parameter.PageSize, if it is less than or equal to zero set to 10, if it is greater than 1000 set to 1000 or else set to parameter.PageSize.
200 FormQuery Creates a BooleanQuery (OR) of two MultiMatchQuery queries for the content fields (title, body, metaDescription, and metaKeywords) one of type CrossFields and one of type PhrasePrefix and assigns it to result.Query.
300 FormFilter Creates a BooleanQuery (AND) of TermQueries to filter on website, language, deviceType, persona and not restricted pages and assigns it to result.Filter.
400 PerformSearch Creates a new SearchRequest using result.Query, result.Filter and result.PageSize, then performs the search and assigns the response to result.SearchResponse.
500 ProcessSearchResponse Creates an IEnumerable of ContentSearchResultsDto from the result.SearchResponse.Document fields (title, pageType, and URL) and assigns it to result.ContentSearchResults.

ProductIndexerPipeline

CreateElasticsearchProduct

Order Pipe Description
100 CreateElasticsearchProduct Create a new ElasticsearchProduct object using the parameter object's fields and assigns it to result.ElasticsearchProduct.
200 SetSpellingCorrection Joins ElasticseachProduct fields (ShortDescription, Content, ManufacturuerItem, Name, Specifications, ErpNumber, CategoryNames, and FilterNames) into a string and assigns it to result.ElasticsearchProduct.SpellingCorrection.

GetIndexableProducts

Order Pipe Description
100 GetIndexableProductsSqlStatementParts Creates a string with the Optimizely default SQL query to get the correct products and fields from the database and assigns it to result.SqlStatement.Assigns result.IncremenalFilter with the proper SQL snippet if parameter.IsIncremental is true.
200 CombineIndexableProductsSqlStatementParts Combines result.SqlStatement, result.CustomFields, and result.IncrementalFilter and assigns it to result.FormattedSqlStatement.
300 PerformIndexableProductsSqlQuery Gets an IEnumerable using the SQL query from result.FormattedSqlStatement and assigns it to result.IndexableProducts.

MapElasticSearchProductProperty

Order Pipe Description
100 MapElasticSearchProductCustomProperties If the parameter.SearchBoost.IsQueryable is true and parameter.PropertyDefinition.IsCustomProperty is true sets result.Property to a new Nest TextProperty.

ProductSearchPipeline

FormProductFilter

Order Pipe Description

100

GetSettings Retrieves SearchGeneralSettings and ProductRestrictionsSettings and assigns them to the result object.
200 FormWebsiteFilter Adds a new MatchQuery using the result.SiteContext.WebsiteDto.Id on the websiteId field to result.AllFilters and assigns it to result.WebsiteFilter.
300 FormLanguageFilter Adds a new MatchQuery using the result.SiteContext.LanguageDto.LanguageCode on the languageCode field to result.AllFilters and assigns it to result.LanguageFilter.
400 FormCategoryFilter Creates a new PrefixQuery to search a single category or the whole category tree based on parameter.ProductSearchParameter fields (SearchCriteria and IncludeSubcategories) then adds it to result.AllFilters and assigns it to result.CategoryFilter.
500 FormRestrictionGroupFilter If Restriction Groups are enabled, uses the ProductSearchPipeline's FormRestrictionGroupFilter pipe to generate a new filter then adds the filter to result.AllFilters and assigns it to result.RestrictionGroupFilter.
600 FormAttributeValueFilter Creates PrefixQueries for attributes based on parameter.ProductSearchParameter.AttributeValueIds and combines them into a BooleanQuery. That BooleanQuery is added to result.AllFilters and assigned to result.AttributeValueFilter.
700 FormAllowedProductsFilter Creates MatchQuery objects for each product in parameter.ProductSearchParameter.AllowedProductIds and combines them into a BooleanQuery. That BooleanQuery is add to result.AllFilters and assigned to result.AllowedProductsFilter.
800 FormPriceRangeFilter

If parameter.ProductSearchParameter.PriceFilters has any values a new MatchQuery is created for each filter and combined into a BooleanQuery (OR) then assigned to result.PriceRangeFilter and added to result.AllFilters.

If parameter.ProductSearchParameter.PriceFilters has no values and parameter.ProductSearchParameter.MinimumPrice or parameter.ProductSearchParameter.MaximumPrice have been supplied a new Range query is created using the supplied min/max values with defaults of 0 and 1000000 if either was omitted. That Range query is then added to result.AllFilters and assigned to result.PriceRangeFilter.

If no filters were supplied and neither the minimumPrice or maximumPrice were supplied nothing is done and the pipe returns.

900 CombineFilters Creates a BoolQuery (AND) of all filters in result.AllFilters and assigns it to result.CombinedFilter.

FormRestrictionGroupFilter

Order Pipe Description
100 GetRestrictionGroupFilterFromCache

Attempts to retrieve the restriction group filter from cache based on the current BillTo/ShipTo and assigns it to result.RestrictionGroupFilter.

If there was a cached filter the pipeline exits.

if there was no cached filter the pipeline continues to the next pipe.

200 GetDefaultVisibilityFilters

Creates a new MatchQuery on the restrictionGroups field to match products with no restriction groups and adds it to result.DefaultVisibilityFilters.

Creates a Terms query on the defaultVisibility field with values of empty and show then adds it to result.DefaultVisibilityFilters.

300 GetWebsiteRestrictionGroupQuery Creates an EF query to get the active restriction groups based on the current website and assigns it to result.WebsiteRestrictionGroupQuery.
400 GetWebsiteRestrictionGroups

Uses result.WebsiteRestrictionGroupQuery to get active restriction groups then assigns them to result.WebsiteRestrictionGroups.

If there are any active restriction groups a BoolQuery (AND) is created from result.DefaultVisibilityFilters and assigned to result.RestrictionGroupFilter then the pipeline exits.

If there are no active restriction groups the pipeline continues to the next pipe.

500 GetCustomerRestrictionGroupIds

If there is no BillTo the pipeline does nothing and continues to the next pipe.

If BillTo or ShipTo are populated result.CustomerRestrictionGroupIds is set to the applicable restriction group IDs from result.WebsiteRestrictionGroupQuery.

600 GetShowOnlyRestrictionGroupIds Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "ShowOnly" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.ShowOnlyRestrictionGroupIds.
700 GetShowRestrictionGroupIds Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "Show" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.ShowRestrictionGroupIds.
800 GetHideRestrictionGroupIds Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "Hide" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.HideRestrictionGroupIds.
900 FormShowOnlySubFilter Creates a Terms query on the field restrictionGroups with the values from result.ShowOnlyRestrictionGroupIds then adds it to result.SubFilters and assigns it to result.ShowOnlySubFilter.
1000 FormShowSubFilter

If there are no filters in result.DefaultVisibilityFilters and no filters in result.ShowRestrictionGroupIds continue to next pipe.

Creates BoolQuery (AND) from the filters in result.DefaultVisibilityFilters.Creates MatchQuery objects on the field restrictionGroups for all restriction groups in result.ShowRestrictionGroupIds.

Combines the previously created queries into a BoolQuery (OR) then adds it to result.SubFilters and assigns it to result.ShowSubFilter.

1100 FormHideSubFilter

If there are no filters in result.HideRestrictionGroupIds continue to next pipe.

Creates MatchQuery (NOT) object on the field restrictionGroups for all restrcition groups in result.HideRestrictionGroupIds.

Combines the previously created queries into a BoolQuery (AND) then adds it to result.SubFilters and assigns it to result.HideSubFilter.

1200 CombineFilters

If result.ShowOnlySubFilter is empty and result.SubFilters is empty continue to next pipe.

Create a BoolQuery (AND) fromt he filters in result.SubFilters.

Combine BoolQuery just created and result.ShowOnlySubFilter into a new BoolQuery (OR) and assign it to result.RestrictionGroupFilter.

1300 AddRestrictionGroupFilterToCache Adds result.RestrictionGroupFilter to the cache based on the current BillTo/ShipTo.

RunProductSearch

Order Pipe Description
100 GetSettings

Retrieves SearchGeneralSettings, CatalogGeneralSettings, FuzzySearchSettings, SponsoredSearchSettings and SearchSuggestionsSettings then assign them to the result object.

Set result.UsingListPrice, result.CanSortPrice, and result.PriceFacetingEnabled from various settings objects.

200 InitializeSearchProperties Retrieve and assign values for result.BoostHelper, result.ElasticsearchIndex, result.ElasticsearchQueryBuilder, result.PhraseSuggestConfiguration, and result.ProductSearchFacetProcessor.
300 FormSortOrder Populate result.SortOrderFields based on parameter.ProductSearchParameter fields.
400 ApplySortOrder Create a new Nest SortDescriptor<ElasticsearchProduct> and set the sort based on result.SortOrderFields.
500 GetQueryFields

Adds the fields that are queryable using data from the search.SearchBoosts table to result.ExactMatchFields, result.PrefixMatchFields, and result.PhraseMatchFields.

Adds the fields that are part of fuzzy match (categoryNames and shortDescription) to result.FuzzyMatchFields.

600 FormSearchCriteriaQuery

If parameter.ProductSearchParameter.SearchCriteria is blank continue on to next pipe without doing anything else.

Creates and assigns queries for result.ExactMatchQuery, result.PrefixMatchQuery, result.PhraseMatchQuery and result.FuzzyMatchQuery (if enabled) based on the fields assigned in GetQueryFields pipe.

All new queries added to result.AllQueries.

700 FormCustomerNameQuery
  • If parameter.ProductSearchParameter.SearchCriteria is blank for BillTo and ShipTo are null continue on to next pipe without doing anything else.
  • If BillTo is not null create a PrefixQuery on the customerNames field and match on BillTo.Id + searchCriteria.
  • If ShipTo is not null create a PrefixQuery on the customerNames field and match on ShipTo.Id + searchCriteria.
  • Create a BoolQuery (OR) from the previously created queries then add it to result.AllQueries and assign it to result.CustomerNameQuery.
800 FormSearchWithinQuery

If parameter.ProductSearchParameter.SearchWithin is blank continue on to the next pipe without doing anything else.

Create a MultiMatchQuery of type CrossFields for Product on the result.ExactMatch fields querying on parameter.ProductSearchParameter.SearchWithin and assign it to result.SearchWithinQuery.

900 CombineQueries

Create a BoolQuery (OR) from result.AllQueries.

If result.SearchWithinQuery is not null create a BoolQuery (AND) with it and the previously created BoolQuery.

Assign the resulting BoolQuery to result.CombinedQuery (either just the first BoolQuery or the combined BoolQuery).

1000 FormProductFilter Use the ProductSearchPipeline's FormProductFilter pipeline to get the product filter and assign it to result.FormProductFilterResult.
1100 FormCategoryAggregation
  • If parameter.ProductSearchParameter.DoFacetedSearches is false continue on to the next pipe without doing anything else.
  • If parameter.ProductSearchParameter.SearchCriteria is blank and parameter.ProductSearchParameter.IncludeSubcategories is false continue on to the next pipe without doing anything else.
  • Using result.FormProductFilterResult.AllFilters create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the categories field and add it to result.AllAggregations and assign it to result.CategoryAggregation.
1200 FormAttributeValueAggregation

If parameter.ProductSearchParameter.DoFacetedSearches is false or result.SearchGeneralSettings.EnableAttributeFilters is false continue on to the next pipe without doing anything else.

If result.SearchGeneralSettings.FilterOnSearch if false and parameter.ProductSearchParameter.SearchCriteria is not blank continue on to the next pipe without doing anything else.

Using result.FormProductFilterResult.AllFilters create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the filters field and add it to result.AllAggregations and assign it to result.AttributeValueAggregation.

If parameter.ProductSearchParameter.GetAllAttributeFacets is false continue on to the next pipe without doing anything else.

Using result.FormProductFilterResult.AllFilters where the filters are not equal to result.FormProductFilterResult.AttribueValueFilter create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the filters field and add it to result.AllAggregations and assign it to result.AllAttributeValueAggregation.

1300 FormPriceRangeAggregation

If parameter.ProductSearchParameter.DoFacetedSearches is false or result.PriceFacetingEnabled is false continue on to the next pipe without doing anything else.

Using result.FormProductFilterResult.AllFilters where the filters are not equal to result.FormProductFilterResult.PriceRangeFilter create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the priceFacet field and add it to result.AllAggregations and assign it to result.PriceRangeAggregation.

1350 FormPersonaScoreFunctionList If parameter.SiteContext.PersonaDtos contains personas with EnableSearchBoost true, the result ScoreFunctions collection will be populated with one WeightFunction for each persona with a TermQuery on the Persona.Id and boost value as a Weight. The ScoreFunctions collection will be added to the query by the ApplyQueryAndFilter pipe.
1400 CombineAggregations

If parameter.ProductSearchParameter.DoFacetedSearches is false continue on to the next pipe without doing anything else.

Call result.CombinedAggregation.Filter for each aggregation in result.AllAggregations and assign the result to result.CombinedAggregation.

1500 FormDidYouMeanSuggestion

If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Call result.PhraseSuggestConfiguration.Configure with productSearchParameter.SearchCriteria and result.SearchSuggestions.DidYouMeanThreshold (limited to 0 to 5) and assign the value to result.DidYouMeanSuggestion and add it to result.AllSuggestions.

1600 FormAutoCorrectSuggestion

If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Call result.PhraseSuggestConfiguration.Configure with productSearchParameter.SearchCriteria and result.SearchSuggestions.AutoCorrectThreshold (limited to 0 to 5) and assign the value to result.DidYouMeanSuggestion and add it to result.AllSuggestions.

1700 CombineSuggestions

If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Call result.CombinedSuggestion.Phrase for each suggestion in result.AllSuggestions and assign the result to result.CombinedSuggestion.

1800 ApplySuggestion

If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Call result.SearchDescriptor.Suggest with result.CombinedSuggestion and assign the result to result.SearchDescriptor.

1900 PerformAggregationSearch

If parameter.ProductSearchParameter.DoFacetedSearches is false or result.AllAggregations is empty continue on to the next pipe without doing anything else.

Create an AggregationSearchDescriptor from result.CombinedQuery and result.FormProductFilterResult.CombinedFilter then use that to query Elasticsearch. Assign result from Elasticsearch to result.AggregationSearchResponse.

2000 ProcessCategoryAggregation Use result.ProductSearchFacetProcessor.ProcessCategoryFacets to process the facets and assign the return value to result.ProductSearchResult.CategoryDtos.
2100 ProcessAttributeValueAggregation  Use ProductSearchFacetProcessor.ProcessAttributeTypeFacets to process the facets and assign the return value to result.ProductSearchResult.AttributeTypeDtos.
2200 ProcessPriceRangeAggregation Use ProductSearchFacetProcessor.ProcessPriceRangeFacets to process the facets and assign the return value to result.ProductSearchResult.PriceRangeDto.
2300 PerformSponsoredSearch

If result.SponsoredSearchSettings.Enabled is false or parameter.IsAutocomplete is true or this is a filtered search or if this is not sorted by relevance or parameter.ProductSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Create a new TermQuery on the isSponsored field. That is added to a new query along with result.CombinedQuery and result.FormProductFilterResult.CombinedFilter with a max result lenght of result.SponsoredSearchSettings.Limit or 5. This is used to query Elasticsearch. The results are assigned to result.SponsoredSearchResponse.
2400 ProcessSponsoredSearchResponse If result.SponsoredSearchResponse is null continue on to the next pipe without doing anything else.

Create a new QueryContainerDescriptor<ElasticsearchProduct> to exclude all sponsored result Ids and assign it to result.ExcludeSponsoredProductsFilter.

Set result.ProductSearchResult.Count to the sponsored search's total results.

If this is the first page of results (result.StartRow equals 0), add the returned products to result.ProductSearchResult.Products and set result.PageSize to result.PageSize minus the number of sponsored results returned.

If this is not the first page of results (result.StartRow does not equal 0), set result.StartRow to result.StartRow minus the number of sponsored results returned.
2500 ApplyQueryAndFilter

If result.ExcludeSponsoredProductsFilter is not null make a new new boolean query with result.FormProductFilterResult.CombinedFilter and result.ExcludeSponsoredProductsFilter.

If result.CombinedQuery is null create a SearchDescriptor using only the above created filter or result.FormProductFilterResult.CombinedFilter.

If result.CombinedQuery is not null and result.SearchGeneralSettings.EnableProductBoost is true, create a SearchDescriptor with a FunctionScore query using result.CombinedQuery and the above created filter or result.FormProductFilterResult.CombinedFilter with a FieldValueFactor function using the boost field and assign it to result.SearchDescriptor.

If result.ScoreFunctions collection is not null or empty, these functions will be added to the FunctionScore query. This will provide boosting of products with certain personas.

If result.CombinedQuery is not null and result.SearchGeneralSettings.EnableProductBoost is false, createa  SearchDescriptor using result.CombinedQuery and the above created filter or result.FormProductFilterResult.CombinedFilter. 
2600 ApplyPaging Set from and size on result.SearchDescriptor using result.StartRow and result.PageSize.
2700 PerformSearch Set from and size on result.SearchDescriptor using result.StartRow and result.PageSize.
2800 ProcessSearchResponse

If result.SearchResponse is null or is not valid return a SearchFailure code with relevant message.

Add returned products from SearchResponse to result.ProductSearchResult.Products.

Add the total count of returned products to result.ProductSearchResult.Count.

If result.CanSortByPrice is false or result.UsingListPrice is false and result.ProductSearchResult.Count is greater than result.CatalogGeneralSettings.PriceSortMaximum, then set result.CanSortByPrice to false, set result.ProductSearchResult.SortOrder to result.SortBy and set result.ProductSearchResult.SortOptions to parameter.SortOptions where PriceLowToHigh and PriceHighToLow are removed.

If result.CanSortByPrice is true or result.UsingListPrice is true and result.ProductSearchResult.Count is less than or equal to result.CatalogGeneralSettings.PriceSortMaximum, then set result.ProductSearchResult.SortOptions to parameter.SortOptions.
2900 ProcessDidYouMeanSuggestion If result.SearchResponse.Suggest is null or result.SearchResponse.Suggest does not contain the key "didyoumean" or didYouMeanSuggestions.Length is not equal to 1 continue on to the next pipe without doing anything else.
Populate result.ProductSearchResult.DidYouMeanSuggestions from the result.SearchResponse.Suggest entry for "didyoumean".
3000 ProcessAutoCorrectSuggestion

If result.SearchResponse.Suggest is null or result.SearchResponse.Suggest does not contain the key "correction" or autoCorrectSuggestions.Length is not equal to 1 continue on to the next pipe without doing anything else.

Populate result.ProductSearchResult.AutoCorrectSuggestion from the result.SearchResponse.Suggest entry for "correction".
Do you find this information helpful? Please log in to provide feedback.

Last updated: Dec 11, 2020

Recommended reading