About Episerver full-text search service

The Episerver Full-Text Search Service (FTS Service) is a stand-alone WCF REST Service on top of an unmodified version of the open source search engine Lucene .NET and uses Atom with extensions described in Atom and Extensions as data protocol. The service is deployed through the EPiServer.Search Nuget package. Services are hosted by the IIS and are invoked by the IndexingService.svc file, which means that multiple services can exists within the same environment. One service also can host multiple indexes as described in Named Indexes and Multi-Search. The following image shows a typical setup:  

Assemblies and namespaces

The EPiServer.Search.IndexingService assembly contains the following namespaces:

• EPiServer.Search.IndexingService contains core classes for the Episerver FTS Service, most notably are the NamedIndex,IndexingService and IndexingServiceHandler classes.
• EPiServer.Search.IndexingService.Configuration contains configuration classes for the Episerver FTS Service.
• EPiServer.Search.IndexingService.Security contains core classes for client authentication
• EPiServer.Search.IndexingService.IndexFieldSerializers contains core classes for serializing and de-serializing Atom feed items and Lucene Document Fields.

Service REST API

The Episerver FTS Service defines the following REST endpoints:

Endpoint	Method	Comment
[baseUri]/update/ ?accessKey={accessKey}	POST	Accepts an Atom feed with feed items to add to the index where: accessKey is the connecting client access key
[baseUri]/search/ ?q={q} &namedindexes={namedindexes} &format=xml &offset={offset} &limit={limit} &accesskey={accesskey}	GET	Returns an Atom feed with feed items corresponding to hits in the index where: q is the query expression to parse namedIndexes is a pipe separated list of indexes to search offset is the offset from first result item limit is the number of items to return from offset accessKey is the connecting client access key
[baseUri]/reset/ ?namedindex={namedindex} &accessKey={accessKey}	POST	Wipes all content for the passed named index
[baseUri]/namedindexes/ ?accesskey={accesskey}	GET	Returns an Atom feed with items corresponding to the configured named indexes for the service

Feature highlights

ReferenceId

When the ReferenceId extension contains a value, the Episerver FTS Service determines that it is not a standalone item, but an item that belongs to another item already in the index, such as comments in Episerver Community. When a comment is added to the index, the ReferenceId is set to the commented entity and when a query expression matched the contents of the comment, the commented entity returns as the IndexResponseItem and not the comment itself.

Any IndexRequestItem with IndexAction.Add and with a ReferenceId set, is added to the reference index automatically created for all configured indexes with the suffix “_ref”. The search engine internally finds the parent item corresponding to the ReferenceId (by ID) and updates it so that all contents of any items in the “_ref” index for the parent item is added to the parent item searchable metadata. In this way, there is no need to send all data over and over when updating the entity client side. You cannot search specific fields in reference data because all default searchable fields chunked together and added to the main items metadata.

You cannot update the reference id property for an item after it is added. All items with the IndexAction Update or Remove automatically updates its parent in the index if it was originally added with the reference ID set.

Content in files are indexed, the installed Ifilters decide which files are included.

DataUri

When the DataUri extension contains a value, the Episerver FTS Service immediately enqueues the whole request (in memory queue). Dequeuening of a DataUri request results in a call to the URI where the contents is concatenated to the IndexRequestItem.DisplayText and IndexRequestItem.MetaData, according to the configured value of maxDisplayTextLength.

The Episerver FTS Service currently only supports File Uri’s and where the file needs to be accessible locally from the service perspective. You can override the behavior by overriding the GetFileUriContent(Uri) method and/or GetNonFileUriContent(Uri) in the IndexingServiceHandler.

Named indexes and multi-search

The Episerver FTS Service lets you configure multiple indexes, which you can use where there is an obvious separation of indexed content, or where the there is an existing (Lucene-compatible) index that is updated from a different source. Multiple named indexes need to be configured so that the fields for the index documents maps to the pre-defined field names in the service. See the Configuring TFS Service topic.

When you update the index, the target index (by name) is specified in the NamedIndex attribute extension of the Atom-formatted request. When you search the index, multiple indexes may be specified in the request and the Episerver FTS Service searches each one of the specified indexes and returns a merged result set. If you do not specify any named index, the default index is used.

VirtualPath

The VirtualPath feature enables structuring of indexed content in a tree structure where you can make searches under a certain tree node. This is accomplished by storing the literal path together with the index document and searching the index with a path with a trailing wildcard. The path always (when updating or searching) includes the full path from the root and up. For example: A document stored with the VirtualPath field set to node1/node2 would be a valid search result when you search for documents with the path node1 or node1/node2. However, it is not considered if the path only specified node2. If the IndexRequestItem.AutoUpdateVirtualPath attribute is set when you update or remove an item, search uses the VirtualPath to update or remove all items under the provided node.

Paging

You can page search results at the service or at the client. Default configuration (useIndexingServicePaging=true) is service paging, and therefore returns only max numbers of items equal to the passed pageSize. Client paging returns max numbers of items equal to the configured maxHitsFromIndexingService (default 500). Consider the client paging option when filter providers are configured, and paging needs to be intact.

Limitations

The Episerver FTS Service handles plain text only and does not understand a markup language, such as HTML, and cannot calculate relevance based on markup (such as <h1> <b> and so on).

The Episerver FTS Service does not crawl the web or update automatically. Indexed content is pushed into the service and thus hands over the responsibility of keeping the index updated to the client.

Third-party search engines

The loose couplings between the FTS Client and Service allows for third-party search engines to implement solutions compatible with the FTS Client independent of platform. The only requirement is to comply with the REST service endpoint specifications. For .NET environments, override the UpdateIndex, GetSearchResults, GetNamedIndexes and ResetNamedIndex methods in the IndexingServiceHandler, thus using the existing WCF REST service implementation.