SearchPhaseExecutionException after updating to EPiServer.Find 12

Mathias J

Vote:

After updating from EPiServer.Find 11.0.0.3701 to 12.3.2.0 I'm getting an error when searching for a page type:

EPiServer.Find.ServiceException: The remote server returned an error: (500) Internal Server Error.
SearchPhaseExecutionException[Failed to execute phase [query], total failure; shardFailures {[cvWFkiGgQQiI7pxso2NfTw][myproject_qa_index][3]: SearchParseException[[myproject_qa_index][3]: query[filtered(ConstantScore(NotDeleted(+cache(CompanyId$$number:[28.0 TO 28.0]) +cache(___types:EPiServer.Core.IContent))))->cache(___types:OpenWeb.Core.Models.Pages.ContactOfficePage)],from[-1],size[1000]: Parse Failure [Failed to parse source [{"size":1000,"query":{"filtered":{"query":{"constant_score":{"filter":{"and":[{"term":{"CompanyId$$number":28}},{"term":{"___types":"EPiServer.Core.IContent"}}]}}},"filter":{"term":{"___types":"OpenWeb.Core.Models.Pages.ContactOfficePage"}}}},"sort":[{"Name$$string.sort":{"missing":"_first","ignore_unmapped":true}}],"fields":["___types","ContentLink.ID$$number","ContentLink.ProviderName$$string","Language.Name$$string"]}]]]; nested: ElasticSearchIllegalArgumentException[Sorting on string type field does not support missing parameter]; }{[cvWFkiGgQQiI7pxso2NfTw][myproject_qa_index][4]: SearchParseException[[myproject_qa_index][4]: query[filtered(ConstantScore(NotDeleted(+cache(CompanyId$$number:[28.0 TO 28.0]) +cache(___types:EPiServer.Core.IContent))))->cache(___types:OpenWeb.Core.Models.Pages.ContactOfficePage)],from[-1],size[1000]: Parse Failure [Failed to parse source [{"size":1000,"query":{"filtered":{"query":{"constant_score":{"filter":{"and":[{"term":{"CompanyId$$number":28}},{"term":{"___types":"EPiServer.Core.IContent"}}]}}},"filter":{"term":{"___types":"OpenWeb.Core.Models.Pages.ContactOfficePage"}}}},"sort":[{"Name$$string.sort":{"missing":"_first","ignore_unmapped":true}}],"fields":["___types","ContentLink.ID$$number","ContentLink.ProviderName$$string","Language.Name$$string"]}]]]; nested: ElasticSearchIllegalArgumentException[Sorting on string type field does not support missing parameter]; }] ---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
   at System.Net.HttpWebRequest.GetResponse()
   at EPiServer.Find.Connection.JsonRequest.GetResponseStream()
   at EPiServer.Find.Api.Command.GetResponse[TResult](IJsonRequest request)
   --- End of inner exception stack trace ---
   at EPiServer.Find.Api.Command.GetResponse[TResult](IJsonRequest request)
   at EPiServer.Find.Api.SearchCommand`1.Execute()
   at EPiServer.Find.Client.Search[TResult](SearchRequestBody requestBody, Action`1 commandAction)
   at EPiServer.Find.SearchExtensions.GetProjectedResult[TResult](ISearch`1 search, SearchContext context)
   at EPiServer.Find.SearchExtensions.GetResult[TResult](ISearch`1 search)
   at EPiServer.Find.Cms.SearchRequestExtensions.GetContentResult[TContentData](ITypeSearch`1 search, Int32 cacheForSeconds, Boolean cacheForEditorsAndAdmins)
   at OpenWeb.Core.Helpers.Search.FindHelper.d__0`1.MoveNext() in c:\Projects\myproject\OpenWeb.Core\Helpers\Search\FindHelper.cs:line 20
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at EPiServer.Filters.FilterForVisitor.Filter(IEnumerable`1 contentItems)
   at OpenWeb.Core.Extensions.FilterForVisitorExtension.FilterOnType[T](IEnumerable`1 collection) in c:\Projects\myproject\OpenWeb.Core\Extensions\FilterForVisitorExtension.cs:line 11
   at OpenWeb.Core.Models.ViewModels.ContactUsPageViewModel.GetOfficeList(String path) in c:\Projects\myproject\OpenWeb.Core\Models\ViewModels\ContactUsPageViewModel.cs:line 295
   at OpenWeb.Core.Models.ViewModels.ContactUsPageViewModel..ctor(ContactUsPage currentPage, Int32 officeId, String path, Boolean usePrefix) in c:\Projects\myproject\OpenWeb.Core\Models\ViewModels\ContactUsPageViewModel.cs:line 58
   at OpenWeb.Web.Controllers.Pages.LocalPageController.Index(LocalPage currentPage, String facet) in c:\Projects\myproject\OpenWeb.Web\Controllers\Pages\LocalPageController.cs:line 186
   at lambda_method(Closure , ControllerBase , Object[] )
   at System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters)
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.b__39(IAsyncResult asyncResult, ActionInvocation innerInvokeState)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`2.CallEndDelegate(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.b__3d()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass33.b__32(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass21.<>c__DisplayClass2b.b__1c()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass21.b__1e(IAsyncResult asyncResult)

ContactUsPageViewModel.cs:

var query = SearchClient.Instance.Search()
		.FilterForVisitor()
		.Filter(page => page.CompanyId.Match(companyIdFromUrl))
		.OrderBy(p => p.Name);

return FilterForVisitorExtension.FilterOnType(FindHelper.GetAllPages(query)
		.Where(s => s.CheckPublishedStatus(PagePublishedStatus.Published))).ToList(); // Line 295

FindHelper.cs:

public static IEnumerable GetAllPages(ITypeSearch query) where T : IContentData
		{
			query = query.Take(1000);

			var batch = query.GetContentResult(); // Line 20, where the exception is thrown
			foreach (var page in batch)
			{
				yield return page;
			}

			var totalNumberOfPages = batch.TotalMatching;

			var nextBatchFrom = 1000;
			while (nextBatchFrom < totalNumberOfPages)
			{
				batch = query.Skip(nextBatchFrom).GetContentResult();
				foreach (var page in batch)
				{
					yield return page;
				}
				nextBatchFrom += 1000;
			}
		}

Another odd thing is that this error doesn't happen with my local developer Find index, only in our stage environment.

#174442

Edited, Jan 26, 2017 13:06

Per Magne Skuseth

Vote:

Did you reindex your staging index after updating Find?

#174446

Jan 26, 2017 13:56

Mathias J

Vote:

Hi Per! Yes, I first ran a custom job which clears the index like this:

var client = ServiceLocator.Current.GetInstance<IClient>();
client.Delete<object>(x => x.GetType().Name.Exists() | !x.GetType().Name.Exists());

After which I ran the built-in job "EPiServer Find Content Indexing Job".

#174447

Edited, Jan 26, 2017 14:00

Per Magne Skuseth

Vote:

There is a "clear index" button in the Find UI, you know :-) Also, the standard indexing job will delete all old IContent that has not been updated. What is going on at "FilterForVisitorExtension.cs:line 11"? It looks a bit like you are both doing FilterForVisitor on the query, and then afterwards as well. That should not be necessary.

#174450

Jan 26, 2017 14:22

Mathias J

Vote:

Ah ok, but I assume the job I ran does the same as the "clear index" button?

FilterForVisitorExtension.cs looks like this:

public static IEnumerable<T> FilterOnType<T>(IEnumerable<IContent> collection) where T : IContent
{
	return EPiServer.Filters.FilterForVisitor.Filter(collection).OfType<T>(); // Line 11
}

#174451

Edited, Jan 26, 2017 14:26

Mathias J

Vote:

Yes you are right, the FilterForVisitor filtering is happening twice. I did however try removing the .FilterForVisitor() in the query but the (same) error still occurs.

#174452

Jan 26, 2017 14:48

Per Magne Skuseth

Vote:

The code looks correct to me. Although you do not need to to run the extra FilterForVisitor after the query has been run.

Looks like the problem is "ElasticSearchIllegalArgumentException[Sorting on string type field does not support missing parameter]", which does not sound right as "name" should never be empty, and even if it was, that should not really be a problem.

#174453

Jan 26, 2017 14:49

Mathias J

Vote:

Now that I try to use the "clear index" button instead of the custom job I get an error message saying:

Unable to load /episerver/Find/proxy/_clear?clear_index=true&clear_stats=false status: 400

#174455

Edited, Jan 26, 2017 14:53

Mathias J

Vote:

It should indeed never be empty as it's not allowed. I realized now that the query in the OP was missing a <ContactOfficePage>, I added that now.

#174457

Jan 26, 2017 14:58

Per Magne Skuseth

Vote:

Perhaps there is something wrong with the index. Could you try the dev index in your staging environment? If there is something wrong with the index, I suggest you contact operations@episerver.com

#174461

Jan 26, 2017 15:19

Mathias J

Vote:

Unfortunately I can't use my dev index in the staging environment as the staging environment is restricted network-wise (getting error message "No connection could be made because the target machine actively refused it" when trying). Is there any way for me to clear the index completely if I log onto the index server? As you're saying, the index might be corrupted somehow, so would be excellent if I can manually clear it by deleting files/folders on the server where the index is located.

#174496

Jan 27, 2017 13:34