dada
Apr 3, 2020
  9399
(3 votes)

Why is my Find indexing job freezing and dying?

I recently had a support case which I spent maybe a few hours too many on.

The symptom was a find indexing job running for about 5 minutes, web app froze up and restarted causing the indexing job to fail prematurly. After looking into application logs, cpu and memory usage (which looked good overall) we could see the app were being restarted .

We did profiling with some hints that the scheduled job thread was aborted. We then took a memory dump 4 minutes into the indexing job.

In the memory dump we could see StackOverflowException, OutOfMemoryException and ThreadAbortException.
After some more digging we found a call to a custom IsSearchable() for the same content over and over.
This behaviour was due to self-referencing content and some custom logic that made it index the same content over and over.


For future cases I'm gonna use something like below to more easily identify errors like this without profiling or memory dumps.
Run the indexing job and check the logs for 'This content was recently indexed' and it might give you a hint on what it's getting stuck on.

[InitializableModule]
[ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
public class InitializationModule2 : IInitializableModule
{

protected static readonly ILog _logger = LogManager.GetLogger(typeof(ContentIndexer));

public void Initialize(InitializationEngine context)
{

    var previouslyIndexedObjects = new Queue<int>();
    int maxLookBack = 1000;

    ContentIndexer.Instance.Conventions.ForInstancesOf<IContent>().ShouldIndex(x =>
    {

        _logger.Info(string.Format("Attempting to index content ID: {0 } Name: {1}", x.ContentLink.ID.ToString(), x.Name.ToString()));

        if (previouslyIndexedObjects.Contains(x.ContentLink.ID))
        {
            _logger.Error(string.Format("WARNING! This content was recently indexed: {0 } Name: {1}", x.ContentLink.ID.ToString(), x.Name.ToString()));
        }

        if (previouslyIndexedObjects.Count() == maxLookBack) { previouslyIndexedObjects.Dequeue(); }
        previouslyIndexedObjects.Enqueue(x.ContentLink.ID);

    return true;
});


...

Apr 03, 2020

Comments

Please login to comment.
Latest blogs
The new LinkItem property in Optimizely CMS, and why it matters

In the past, we have used different tricks to achieve this. Now, the LinkItem property is finally built-in in Optimizely CMS 12!

Tomas Hensrud Gulla | Mar 20, 2023 | Syndicated blog

A day in the life of an Optimizely Developer - Vertical Slicing in CMS12

There is such a vast choice these days in how you can build a website, aside from all of the different programming languages out there, there are...

Graham Carr | Mar 20, 2023

Tag Helpers in CMS 12

Microsoft introduced the TagHelper back in the primordial soup of ASP.NET vNext which became .Net 5, then .NET Core then .Net 5 you know the drill…...

MartinOttosen | Mar 20, 2023

Security.txt with minimal api .net core

Learn why security.txt is a good practice to implement and how easy it can be done.

Luc Gosso (MVP) | Mar 20, 2023 | Syndicated blog