(429) Too Many Requests?

swc
swc
Vote:
 

We subscribe to EpiServer Find, and recently tried to index our site for the first time.  We get this error in the message of the find content indexing job:

Indexing job [SiteName] [content]: Reindexing completed.
ExecutionTime: 2minutes 41seconds
Number of content indexed: 1701
Number of content errors: 0
Number of batch errors: 1220
An exception occured while indexing (Batch): The remote server returned an error: (429) Too Many Requests. 

Too many requests (see log for more information) 

I look in the log, but all I see is the same exception.  I assume something isn't configured right, but I'm not sure what.  Any suggestions?

Thanks!

#90073
Sep 02, 2014 3:10
Vote:
 

There are a limit in the amount of request per second (see more here: http://www.episerver.se/hosting/order-episerver-find/).

You have propably hit that limit. If it is in the main indexing job, it should be handled by code so it might be something to report as a bug to EPiServer

#90079
Sep 02, 2014 7:20
Vote:
 

Have you changed the ContentBatchSize? By default it is 100 but considering the number of batches you trtied to send (1220) indicates that it might have been set to a lower number.

Also what version are you running?

/Henrik

#90089
Sep 02, 2014 9:11
Vote:
 

Henrik. I did not know of this setting, where do you set it if you would like to change it?

#90091
Sep 02, 2014 9:38
Vote:
 

Also... What's the traffic like and how much are you using Find in the templates?

Either way it's good practise to chain .StaticallyCacheFor(TimeSpan.FromMinutes(x)) to your queries.

#90092
Sep 02, 2014 9:39
Vote:
 

Henrik:
You can change the BatchSize by setting it on the ContentIndexer:

var contentIndexer = ServiceLocator.Current.GetInstance();
contentIndexer.ContentBatchSize = 500;

#90104
Sep 02, 2014 10:50
swc
Vote:
 

So it was indeed the batch size.  A developer was bumping into the index request size limit so they decreased the batch size, which limited the size of the individual requests, but at the cost of running into the request limit.  Adjusting back up got around this issue.  

Thanks!

#90122
Sep 02, 2014 13:24
This topic was created over six months ago and has been resolved. If you have a similar question, please create a new topic and refer to this one.
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.