AI OnAI Off

Issue with Episerver Find CMS Reindexing Scheduled Job

hiral shah
hiral shah
Vote:
 

Hi,

We are currently facing constant issues of site going down for almost on everyday for Production Site. This happenes almost at every 3 hours of time gap which is the duration set to run the Episerver.Find Reindexing job on the CMS. 

  • The two page Files which are mentioned in the below log Error, they do not present on the CMS. 
  • When i try to execute the indexing job manually i see no error but when it happns automatically i see below errors and continuous outages. 
  • The Episerver version is Episerver CMS 7.0 and Episerver.Find.CMS  DLL version is 7.0.586.0
2017-07-03 00:10:48,287 [32] ERROR EPiServer.Global: 1.2.5 Unhandled exception in ASP.NET
System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> EPiServer.Find.ServiceException: The remote server returned an error: (400) Bad Request.
SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[BQ_eWjzaR3W9LrRgbRnYaQ][aquaproducts_index50292][0]: RemoteTransportException[[sefindprod02esd04][inet[/10.3.151.4:9300]][search/phase/query]]; nested: SearchParseException[[aquaproducts_index50292][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"filtered":{"query":{"filtered":{"query":{"query_string":{"fields":["Name$$string.standard","Title$$string.standard","SubTitle$$string.standard","Teaser$$string.standard","OverrideNavigationTitle$$string.standard","BrowserTitle$$string.standard","MetaKeywords$$string.standard","MainContent.AsViewedByAnonymous$$string.standard"],"query":"\\[url=https\\:\\/\\/center.renthop.net\\/townhouses\\/4939\\-houses\\-to\\-rent\\-in\\-county\\-down\\-northern\\-ireland.html\\]houses to rent in county down northern ireland\\[/url\\]"}},"filter":{"term":{"LanguageBranch$$string":"en"}}}},"filter":{"term":{"___types":"aquabots.com.Page_Types.Aquabots.Generic.GenericPage"}}}},"fields":["___types","PageLink.ID$$number","PageLink.ProviderName$$string"]}]]]; nested: QueryParsingException[[aquaproducts_index50292] Failed to parse query [\[url=https\:\/\/center.renthop.net\/townhouses\/4939\-houses\-to\-rent\-in\-county\-down\-northern\-ireland.html\]houses to rent in county down northern ireland\[/url\]]]; nested: ParseException[Cannot parse '\[url=https\:\/\/center.renthop.net\/townhouses\/4939\-houses\-to\-rent\-in\-county\-down\-northern\-ireland.html\]houses to rent in county down northern ireland\[/url\]': Lexical error at line 1, column 170.  Encountered:  after : "/url\\]"]; nested: TokenMgrError[Lexical error at line 1, column 170.  Encountered:  after : "/url\\]"]; }{[vwhC_QFVRSCt3Iu4HihV1A][aquaproducts_index50292][1]: SearchParseException[[aquaproducts_index50292][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"filtered":{"query":{"filtered":{"query":{"query_string":{"fields":["Name$$string.standard","Title$$string.standard","SubTitle$$string.standard","Teaser$$string.standard","OverrideNavigationTitle$$string.standard","BrowserTitle$$string.standard","MetaKeywords$$string.standard","MainContent.AsViewedByAnonymous$$string.standard"],"query":"\\[url=https\\:\\/\\/center.renthop.net\\/townhouses\\/4939\\-houses\\-to\\-rent\\-in\\-county\\-down\\-northern\\-ireland.html\\]houses to rent in county down northern ireland\\[/url\\]"}},"filter":{"term":{"LanguageBranch$$string":"en"}}}},"filter":{"term":{"___types":"aquabots.com.Page_Types.Aquabots.Generic.GenericPage"}}}},"fields":["___types","PageLink.ID$$number","PageLink.ProviderName$$string"]}]]]; nested: QueryParsingException[[aquaproducts_index50292] Failed to parse query [\[url=https\:\/\/center.renthop.net\/townhouses\/4939\-houses\-to\-rent\-in\-county\-down\-northern\-ireland.html\]houses to rent in county down northern ireland\[/url\]]]; nested: ParseException[Cannot parse '\[url=https\:\/\/center.renthop.net\/townhouses\/4939\-houses\-to\-rent\-in\-county\-down\-northern\-ireland.html\]houses to rent in county down northern ireland\[/url\]': Lexical error at line 1, column 170.  Encountered:  after : "/url\\]"]; nested: TokenMgrError[Lexical error at line 1, column 170.  Encountered:  after : "/url\\]"]; }] ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.
   at System.Net.HttpWebRequest.GetResponse()
   at EPiServer.Find.Connection.JsonRequest.GetResponse()
   at EPiServer.Find.Api.Command.GetResponse[TResult](IJsonRequest request)
   --- End of inner exception stack trace ---
   at EPiServer.Find.Api.Command.GetResponse[TResult](IJsonRequest request)
   at EPiServer.Find.Api.SearchCommand`1.Execute()
   at EPiServer.Find.Client.Search[TSource](SearchRequestBody requestBody, Action`1 commandAction)
   at EPiServer.Find.SearchExtensions.GetProjectedResult[TResult](ISearch`1 search, SearchContext context)
   at EPiServer.Find.SearchExtensions.GetResult[TResult](ISearch`1 search)
   at EPiServer.Find.Cms.SearchRequestExtensions.GetPagesResult[TPageData](ITypeSearch`1 search, LanguageSelector languageSelector)
   at aquabots.com.Templates.Aquabots.Units.Global.SiteSearchListing.BindDetails()
   at aquabots.com.Templates.Aquabots.Units.Global.SiteSearchListing.Page_Load(Object sender, EventArgs e)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.HandleError(Exception e)
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.ProcessRequest()
   at System.Web.UI.Page.ProcessRequest(HttpContext context)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
2017-07-03 00:35:41,357 [75] ERROR EPiServer.Find.Cms.ContentIndexer: An exception occured while indexing (Batch): You are not authorized to access /PageFiles/2463/. 
System.UnauthorizedAccessException: You are not authorized to access /PageFiles/2463/
   at EPiServer.Web.Hosting.VirtualPathVersioningProvider.GetDirectory(String virtualPath)
   at EPiServer.Web.Hosting.VirtualPathVersioningProvider.GetDirectory(String virtualPath)
   at EPiServer.Web.Hosting.VirtualPathVersioningProvider.GetDirectory(String virtualPath)
   at EPiServer.Web.Hosting.VirtualPathNonUnifiedProvider.GetDirectory(String virtualDir)
   at EPiServer.Web.Hosting.VirtualPathNonUnifiedProvider.GetDirectory(String virtualDir)
   at EPiServer.Web.Hosting.VirtualPathNonUnifiedProvider.GetDirectory(String virtualDir)
   at EPiServer.Web.Hosting.VirtualPathNonUnifiedProvider.GetDirectory(String virtualDir)
   at EPiServer.Find.Cms.ContentExtensions.GetPageFiles(PageData page)
   at EPiServer.Find.Cms.ContentIndexer.Index(IEnumerable`1 content, IndexOptions options)
   at EPiServer.Find.Cms.ContentIndexer.ReIndex(Action`1 statusAction, Func`1 isStopped)
2017-07-03 00:35:44,976 [75] ERROR EPiServer.Find.Cms.ContentIndexer: An exception occured while indexing (Batch): You are not authorized to access /PageFiles/2255/. 
System.UnauthorizedAccessException: You are not authorized to access /PageFiles/2255/
   at EPiServer.Web.Hosting.VirtualPathVersioningProvider.GetDirectory(String virtualPath)
   at EPiServer.Web.Hosting.VirtualPathVersioningProvider.GetDirectory(String virtualPath)
   at EPiServer.Web.Hosting.VirtualPathVersioningProvider.GetDirectory(String virtualPath)
   at EPiServer.Web.Hosting.VirtualPathNonUnifiedProvider.GetDirectory(String virtualDir)
   at EPiServer.Web.Hosting.VirtualPathNonUnifiedProvider.GetDirectory(String virtualDir)
   at EPiServer.Web.Hosting.VirtualPathNonUnifiedProvider.GetDirectory(String virtualDir)
   at EPiServer.Web.Hosting.VirtualPathNonUnifiedProvider.GetDirectory(String virtualDir)
   at EPiServer.Find.Cms.ContentExtensions.GetPageFiles(PageData page)
   at EPiServer.Find.Cms.ContentIndexer.Index(IEnumerable`1 content, IndexOptions options)
   at EPiServer.Find.Cms.ContentIndexer.ReIndex(Action`1 statusAction, Func`1 isStopped)
2017-07-03 00:42:56,771 [11] ERROR EPiServer.Global: 1.2.5 Unhandled exception in ASP.NET
System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at BVNetwork.Bvn.FileNotFound.Logging.Logger.LogRequests(List`1 logEvents)
   at BVNetwork.Bvn.FileNotFound.Logging.Logger.LogRequest(String oldUrl, String referer)
   at BVNetwork.FileNotFound.NotFoundPageUtil.HandleOnLoad(Page page, Uri urlNotFound, String referer)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.HandleError(Exception e)
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.ProcessRequest()
   at System.Web.UI.Page.ProcessRequest(HttpContext context)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
2017-07-03 01:36:40,517 [34] ERROR EPiServer.Global: 1.2.5 Unhandled exception in ASP.NET

Please suggest here what is going on. 

Thanks,

Hiral shah

#180316
Jul 05, 2017 21:25
K Khan
K Khan
Vote:
 

Automatic scheduled jobs are not executed on a web request. Therfore under Schedule Job, HttpContext.Current will be null, Indexing job will not have any user rights during run. The manual trigger will cause the task to run with the currently logged in user’s account and with access to that HttpContext. If you do not require those objects getting indexed, exclude them by using Find Conventions. Otherwise under the version you mentioned most probably you may require to build custom job to cover access issues in your scenario.

/K

#180317
Edited, Jul 05, 2017 21:40
hiral shah
hiral shah
Vote:
 

Hi Khan,

Thanks for responding back. I have 2 questions though.

1) how does these errors responsible to take the site down for  couple minutes? 

2) If you notice i see 400 bad request in the http unhandled exception. Does it because of the httpcpntext.currnt value is null?

  • Do you think it is because of the find.cms version i used in my application has some of the bugs? Does upgrading that solve this ?
  • Also regrading those page file IDs, they are no longer be part of the CMS therefore i am not sure of what page types they are. IS there any other way to exclude them from indexing if the pages are deleted or unpublished ?

Thanks,

Hiral

#180363
Edited, Jul 07, 2017 15:56
K Khan
K Khan
Vote:
 

I am afraid, based on the exception, I cannot make any relation of site outages. After stopping this service, was your site working ok? There are hunderds of reasons (mostly memory leaks) I have seen, sometime just badly structured Top menu was the reason of these kind of periodic outages. In most of the cases EPiServer was very robust but custom logics effects performance.

Generally I will highly recommend to stay updated as updates contains not only new features but also fixes regarding performance and security. Updates doesn't guranties that your site issue will get resolved untill we know the reason. Have a look on release notes, may you find some hint.

/K

#180374
Edited, Jul 07, 2017 17:33
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.