Pause/disable automatic indexing

Vote:
 

Hi everybody,

We have a scheduled job that imports external PIM data to Episerver Commerce. From time to time there's alot of content beeing imported and sometimes the job crasches due to a deadlock between the import job and the automatic indexing that is performed by Episerver Find.

I wonder if there's a way to programmatically disable/pause automatic indexing that normally occur when content is changed. It can be done in configuration as described here but I need to do it temporarly from code during execution of the import job. 

Thank you,

Tony

#230788
Edited, Nov 13, 2020 10:52
Vote:
 

There is a change recently which should allow you to stop indexing catalog content programmatically, unfortunately that change is not yet released. I will ping Commerce team about this. 

#230792
Nov 13, 2020 14:48
Vote:
 

Hi Tony you should be able to run the following to disable automatic indexing.

EventedIndexingSettings.Instance.EventedIndexingEnabled = false;
EventedIndexingSettings.Instance.ScheduledPageQueueEnabled = false;

This stops the content events from triggering Find.  You will need to add some logic to manually index your items and then re-enable event indexing.

#231063
Nov 18, 2020 16:15
Vote:
 

it is not enough I am afraid. Find will still index catalog content from lower level events, unless you disable that as well

#231065
Nov 18, 2020 16:23
Vote:
 

Thank you for your answers!

I've made a simple, temporary solution that works fine while we're waiting for the release of a permanent solution by the Commerce team.

A bool object is stored in cache when I want to disable indexing and then removes it when it shoult be enabled again. I use the cache approach simply to let it time out after a specified time (30 min) in case enabling the indexing isn't done for some reason.

I then let a custom indexing convetion check wether the cache object exists or not. If it exist indexing will be disabled.

ContentIndexer.Instance.Conventions
    .ForInstancesOf<IContent>()
    .ShouldIndex(x => !x.IsIndexingPaused());

When the import of Commerce content is done I enable indexing again and programmatically start the "EPiServer Find Content Indexing Job" to index any content that has been changed while the indexing was disabled.

#231068
Edited, Nov 18, 2020 19:51
Quan Mai - Nov 19, 2020 8:31
Creative!
Vote:
 

I have a good conversation with my colleague Karl and I stand corrected. 

EventedIndexingSettings.Instance.ScheduledPageQueueEnabled = false;

should be enough to stop the indexing triggered by lower level events. The items will still be added to the queue, but the indexer will not process it until it's turned true. However with the new change in Find.Commerce, you can turn off the part that adding items to queue, which itself is very heavy. 

#231115
Nov 19, 2020 10:42
Vote:
 

That's what we have running in production.

We have had that solution in place for a little while now on a retailer with a very large catalog (>600,00 items).  When the system does daily synch with the ERP we run this and then switch it back.  We are ensuring items are batch indexed manually though.

#231116
Edited, Nov 19, 2020 10:53
Quan Mai - Nov 19, 2020 10:54
Yes I know you said that, and that's what " I stand corrected" part for ;)
- Nov 19, 2020 11:04
All is forgiven Quan :)
Vote:
 

We have been using custom scheduler job, which triggers Epi Find in background. Sometimes this exceeds the QPS limit of an index. I was thinking to disable automatic indexing, then add some logic to manually index the items. And then enable automatic find index.

If I do not apply manual logic and just enable automatic find index after completion of scheduler job. Would Epi Find able to re-index item in batches to prevent too many requests?

#253122
Apr 13, 2021 1:37
Vote:
 

Hi Raj,

Depends on your situation.  If you are still calling IClient.Index() within your custom code, then the items are being indexed.  What you are not needing is the queue to populate and the event indexer to duplicate the same thing again.  That would be a possibility to try the above recommendation in this thread if you are having problems with a large bulk operation causing many events to queue and erroring.

Otherwise, simply have your content items saved via ContentRepository, etc. and allow the automatic event indexer to do its thing.  No need to manually Index in Find. You could look at using a service queue for better performance?  https://world.episerver.com/documentation/developer-guides/search-navigation/NET-Client-API/Indexing/#IndexingLocalServiceQueue

#253142
Edited, Apr 13, 2021 10:00
Vote:
 

If you can, index in batch, use Index(IEnumerable<IContent> ...) to reduce the request to Find.

#253143
Apr 13, 2021 10:11
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.