We have a scheduled job that imports external PIM data to Episerver Commerce. From time to time there's alot of content beeing imported and sometimes the job crasches due to a deadlock between the import job and the automatic indexing that is performed by Episerver Find.
I wonder if there's a way to programmatically disable/pause automatic indexing that normally occur when content is changed. It can be done in configuration as described here but I need to do it temporarly from code during execution of the import job.
There is a change recently which should allow you to stop indexing catalog content programmatically, unfortunately that change is not yet released. I will ping Commerce team about this.
Hi Tony you should be able to run the following to disable automatic indexing.
EventedIndexingSettings.Instance.EventedIndexingEnabled = false;EventedIndexingSettings.Instance.ScheduledPageQueueEnabled = false;
EventedIndexingSettings.Instance.EventedIndexingEnabled = false;
EventedIndexingSettings.Instance.ScheduledPageQueueEnabled = false;
This stops the content events from triggering Find. You will need to add some logic to manually index your items and then re-enable event indexing.
it is not enough I am afraid. Find will still index catalog content from lower level events, unless you disable that as well
Thank you for your answers!
I've made a simple, temporary solution that works fine while we're waiting for the release of a permanent solution by the Commerce team.
A bool object is stored in cache when I want to disable indexing and then removes it when it shoult be enabled again. I use the cache approach simply to let it time out after a specified time (30 min) in case enabling the indexing isn't done for some reason.
I then let a custom indexing convetion check wether the cache object exists or not. If it exist indexing will be disabled.
.ShouldIndex(x => !x.IsIndexingPaused());
When the import of Commerce content is done I enable indexing again and programmatically start the "EPiServer Find Content Indexing Job" to index any content that has been changed while the indexing was disabled.
I have a good conversation with my colleague Karl and I stand corrected.
should be enough to stop the indexing triggered by lower level events. The items will still be added to the queue, but the indexer will not process it until it's turned true. However with the new change in Find.Commerce, you can turn off the part that adding items to queue, which itself is very heavy.
That's what we have running in production.
We have had that solution in place for a little while now on a retailer with a very large catalog (>600,00 items). When the system does daily synch with the ERP we run this and then switch it back. We are ensuring items are batch indexed manually though.
We have been using custom scheduler job, which triggers Epi Find in background. Sometimes this exceeds the QPS limit of an index. I was thinking to disable automatic indexing, then add some logic to manually index the items. And then enable automatic find index.
If I do not apply manual logic and just enable automatic find index after completion of scheduler job. Would Epi Find able to re-index item in batches to prevent too many requests?
Depends on your situation. If you are still calling IClient.Index() within your custom code, then the items are being indexed. What you are not needing is the queue to populate and the event indexer to duplicate the same thing again. That would be a possibility to try the above recommendation in this thread if you are having problems with a large bulk operation causing many events to queue and erroring.
Otherwise, simply have your content items saved via ContentRepository, etc. and allow the automatic event indexer to do its thing. No need to manually Index in Find. You could look at using a service queue for better performance? https://world.episerver.com/documentation/developer-guides/search-navigation/NET-Client-API/Indexing/#IndexingLocalServiceQueue
If you can, index in batch, use Index(IEnumerable<IContent> ...) to reduce the request to Find.