(429) Too Many Requests?

swc

Vote:

We subscribe to EpiServer Find, and recently tried to index our site for the first time. We get this error in the message of the find content indexing job:

Indexing job [SiteName] [content]: Reindexing completed.
ExecutionTime: 2minutes 41seconds
Number of content indexed: 1701
Number of content errors: 0
Number of batch errors: 1220
An exception occured while indexing (Batch): The remote server returned an error: (429) Too Many Requests.

Too many requests (see log for more information)

I look in the log, but all I see is the same exception. I assume something isn't configured right, but I'm not sure what. Any suggestions?

Thanks!

#90073

Sep 02, 2014 3:10

Henrik Fransas

Vote:

There are a limit in the amount of request per second (see more here: http://www.episerver.se/hosting/order-episerver-find/).

You have propably hit that limit. If it is in the main indexing job, it should be handled by code so it might be something to report as a bug to EPiServer

#90079

Sep 02, 2014 7:20

Henrik Lindström

Vote:

Have you changed the ContentBatchSize? By default it is 100 but considering the number of batches you trtied to send (1220) indicates that it might have been set to a lower number.

Also what version are you running?

/Henrik

#90089

Sep 02, 2014 9:11

Henrik Fransas

Vote:

Henrik. I did not know of this setting, where do you set it if you would like to change it?

#90091

Sep 02, 2014 9:38

Vote:

Also... What's the traffic like and how much are you using Find in the templates?

Either way it's good practise to chain .StaticallyCacheFor(TimeSpan.FromMinutes(x)) to your queries.

#90092

Sep 02, 2014 9:39

Henrik Lindström

Vote:

Henrik:
You can change the BatchSize by setting it on the ContentIndexer:

var contentIndexer = ServiceLocator.Current.GetInstance();
contentIndexer.ContentBatchSize = 500;

#90104

Sep 02, 2014 10:50

swc

Vote:

So it was indeed the batch size. A developer was bumping into the index request size limit so they decreased the batch size, which limited the size of the individual requests, but at the cost of running into the request limit. Adjusting back up got around this issue.

Thanks!

#90122

Sep 02, 2014 13:24