Episerver Find Content Indexing Job Issue

Ram Natarajan

Vote:

Hi,

We have a very content heavy site close to 85000 entries. Since we upgraded the episerver versions to our site, the epi find job hasn't been behaving as it used to be. It either looses its full indexes or does incomplete indexing. The only common thing that occurs is the below error in the log after which the find job resets itself and starts again and does half a job. This issue has been eating my brain for a while now. Has anyone come across this issue? Help is very much appreciated.

2016-04-13 15:31:20,719 [703] ERROR EPiServer.Find.Cms.ContentIndexer - 758009-STGING1: An exception occurred while indexing (Batch): The request was aborted: The request was canceled. 
System.Net.WebException: The request was aborted: The request was canceled. ---> System.IO.IOException: Cannot close stream until all bytes are written.
   at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
   --- End of inner exception stack trace ---
   at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
   at System.Net.ConnectStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.ConnectStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at EPiServer.Find.Json.Serializer.SerializeObjectsToJsonRequest(JsonSerializer serializer, IJsonRequest jsonRequest, IEnumerable values)
   at EPiServer.Find.Api.BulkCommand.Execute()
   at EPiServer.Find.Cms.ContentIndexer.IndexWithRetry(IContent[] contents, Int32 maxRetries)
   at EPiServer.Find.Cms.ContentIndexer.Index(IEnumerable`1 content, IndexOptions options)
   at EPiServer.Find.Cms.ContentIndexer.<>c__DisplayClass19.b__e(IEnumerable`1 currentBatch, ParallelLoopState loopstate)
System.Net.WebException: The request was aborted: The request was canceled. ---> System.IO.IOException: Cannot close stream until all bytes are written.
   at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
   --- End of inner exception stack trace ---
   at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
   at System.Net.ConnectStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.ConnectStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at EPiServer.Find.Json.Serializer.SerializeObjectsToJsonRequest(JsonSerializer serializer, IJsonRequest jsonRequest, IEnumerable values)
   at EPiServer.Find.Api.BulkCommand.Execute()
   at EPiServer.Find.Cms.ContentIndexer.IndexWithRetry(IContent[] contents, Int32 maxRetries)
   at EPiServer.Find.Cms.ContentIndexer.Index(IEnumerable`1 content, IndexOptions options)
   at EPiServer.Find.Cms.ContentIndexer.<>c__DisplayClass19.b__e(IEnumerable`1 currentBatch, ParallelLoopState loopstate)

Many thanks ahead,

Ram

#147405

Apr 13, 2016 17:15

Ram Natarajan

Vote:

This is the error in the history of the job:

Indexing job [dk] [content]: Reindexing completed. ExecutionTime: 28 minutes 37 seconds Number of contents indexed: 65959 Number of content errors: 0 Number of batch errors: 2
758009-STGING1: An exception occurred while indexing (Batch): The request was aborted: The request was canceled. (see log for more information) 
758009-STGING1: An exception occurred while indexing (Batch): The request was aborted: The request was canceled. (see log for more information) 

Number of external url best bets indexed: 1Indexing job [Global assets and other data] [content]: Reindexing completed. ExecutionTime: 28 minutes 32 seconds Number of contents indexed: 65025
Number of external url best bets indexed: 1

#147406

Apr 13, 2016 17:22

Daniel Ovaska

Vote:

I would set log level to all for Find and try to pin point the error. Maybe large files and slow connection? Just guessing wildly. Need more logs I think then probably a support ticket. Sounds like a timeout issue from the little information so far. Might be something with max file size maybe?

<logger name="EPiServer.Find">
<appender-ref ref="FindRollingLogFileAppender" />
<level value="All" />
</logger>

Plus configure appended...

#147411

Edited, Apr 13, 2016 20:02

Ram Natarajan

Vote:

Hi Daniel,

I have put a trace on the find job and have uploaded the log below.

Find All Log: http://www.dk.com/uk1/EpiFindAll.txt

Find Debug Log:http://www.dk.com/uk1/EpiFindDebug.txt

By the way how to create a support ticket?

Regards,
Ram

#147479

Apr 15, 2016 13:18

Daniel Ovaska

Vote:

My best guess is that this is caused by large files (or large batches) that is getting cut for some reason. Might be Find max file size (50Mb I think) at other end or at your end. SSL and firewalls might be possible causes as well in that case.

Might want to try reducing the maximum size of file that should be indexed and the batch size that is being sent to Find and see if that helps. Otherwise contact their support and they will help

Check out

http://antecknat.se/blog/2015/02/23/convention-for-episerver-find-to-ignore-large-files/

for some ideas of how to create conventions for limiting file size and batch size.

If you are partner you can add support tickets here

http://world.episerver.com/support/

#147482

Edited, Apr 15, 2016 15:01