Episerver Find persistent queue

As part of the continuous effort to operate optimally in the cloud, we have done multiple things to adapt the Episerver platform.

In Episerver Find 12.1.0, we changed a fundamental part of indexing to better work in a cloud environment, where several client instances may be present and there is less control of the actual computing power. Previously, a document to be indexed was put into a memory queue and processed by the client based in order of submission. In a cloud environment (such as Azure), this approach may cause issues. For example, an instance is stopped due to scaling or maintenance. In this case, the information in memory is lost, and those documents are not indexed. Also, when using a memory queue, the load of indexing could not be split across multiple instances unless the indexing was initiated on different machines.

To solve these issues, the memory queue approach has been re-implemented using a persistent queue that stores requests and events in a SQL database. This way, data is maintained more reliably, and queue information is accessible from all machines in a cluster. There will be no data dropped if a machine is stopped. As an additional benefit, we made sure that, if an item is submitted repeatedly but no changes are present in subsequent data sets, it is not indexed. Finally, we added the ability to trace failed events. See also: Tracing events in Episerver Find