dada
Apr 3, 2020
  9085
(3 votes)

Why is my Find indexing job freezing and dying?

I recently had a support case which I spent maybe a few hours too many on.

The symptom was a find indexing job running for about 5 minutes, web app froze up and restarted causing the indexing job to fail prematurly. After looking into application logs, cpu and memory usage (which looked good overall) we could see the app were being restarted .

We did profiling with some hints that the scheduled job thread was aborted. We then took a memory dump 4 minutes into the indexing job.

In the memory dump we could see StackOverflowException, OutOfMemoryException and ThreadAbortException.
After some more digging we found a call to a custom IsSearchable() for the same content over and over.
This behaviour was due to self-referencing content and some custom logic that made it index the same content over and over.


For future cases I'm gonna use something like below to more easily identify errors like this without profiling or memory dumps.
Run the indexing job and check the logs for 'This content was recently indexed' and it might give you a hint on what it's getting stuck on.

[InitializableModule]
[ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
public class InitializationModule2 : IInitializableModule
{

protected static readonly ILog _logger = LogManager.GetLogger(typeof(ContentIndexer));

public void Initialize(InitializationEngine context)
{

    var previouslyIndexedObjects = new Queue<int>();
    int maxLookBack = 1000;

    ContentIndexer.Instance.Conventions.ForInstancesOf<IContent>().ShouldIndex(x =>
    {

        _logger.Info(string.Format("Attempting to index content ID: {0 } Name: {1}", x.ContentLink.ID.ToString(), x.Name.ToString()));

        if (previouslyIndexedObjects.Contains(x.ContentLink.ID))
        {
            _logger.Error(string.Format("WARNING! This content was recently indexed: {0 } Name: {1}", x.ContentLink.ID.ToString(), x.Name.ToString()));
        }

        if (previouslyIndexedObjects.Count() == maxLookBack) { previouslyIndexedObjects.Dequeue(); }
        previouslyIndexedObjects.Enqueue(x.ContentLink.ID);

    return true;
});


...

Apr 03, 2020

Comments

Please login to comment.
Latest blogs
Optimizely shortcuts in CMS 12 will get a trailing slash appended!

Not all URLs will work when the trailing slash is added, and that could cause problems. Hopefully it will be fixed soon.

Tomas Hensrud Gulla | Nov 26, 2022 | Syndicated blog

One week left of the beta certification period

Today it's one week until the last beta certification day. Do you exam no later than Wednesday the 30th.  Here are the reference and exam guides:...

Karen McDougall | Nov 23, 2022

Unbelievable FREE Heatmapping With Optimizely Web

Within this guide, you will learn how to integrate a free heat mapping tool to Optimizely Web so you can turbocharge your A/B testing capabilities....

Jon Jones | Nov 22, 2022 | Syndicated blog

Import Your Localized Resources in v7.5

Import functionality has been delayed for several versions of DbLocalizationProvider. However, with the help of community member - this feature has...

valdis | Nov 21, 2022 | Syndicated blog