Search statistics are part of the core ElasticSearch instance that's hosted as a service, they aren't held in the database.
You can see this as when you hit the search stats area in the admin, you'll notice it making calls to a proxy URL e.g
/EPiServer/Find/proxy/_stats/query?from=2024-01-11T11%3A00%3A00Z&to=2024-01-12T
Any of the /proxy/ calls are actually just forwarded directly to the service as they are just standard elasticsearch queries and none of the laters on top.
Statistics will show up as long as you're using the track call at the end of your search code that tells the service to track this information. Also not everything is tracked, things like autocomplete and did you mean are only tracked if they have hits (e.g navigations to the page) and there's a 90 day retention on that data. There's no configuration as it's just part of how elasticsearch works.
So the key to keeping this is going to be making sure you're on the same find index. With a CMS 12 .NET Core migration you get new DXP infrastructure, I think on some of ours this included a new s&n index but that might have just been a cluster move for performance. So check with your CSM and technical reps to make sure, there's no way to transfer so it'll only work that way.
Although you might be able to write a job or code to fool it and repop after deployment with quieres and hit tracking.
I just checked with Optimizely and they said you should be able to keep the existing index, therefore I'd expect stats to work
Statistics is not in your DB but rather in your index - whenever you call .Track() on your query it will send that to the server. so as long as you keep the same index and if you don't clear statistics, you will be able to keep it when you "import"/upgrade your DB
Thank you Scott for your detailed answer, and Quan for following up.
Our code has and always had .Track() as part of the search queries.
However, within our INTE & PREP environments, no new statistics are being registered. Since it should be automatic, what could prevent them from being added?
Also, how can I make sure I keep the same index during migration?
I hadn't recently debugged our search function, but decided to try. I have a SQL Server database I exported to use for localhost development, which I'm sure if it makes a difference or not.
It turns out that it is throwing and catching exceptions when trying to track statistics. I am using all the latest .NET 6 Epi packages. Debugging this, I tracked this error to:
Sample log entry:
2024-01-12T20:28:25.556660236Z fail: EPiServer.Find.Personalization.PersonalizationTrackConventions[0]
2024-01-12T20:28:25.556667736Z PersonalizationTrackConventions could not get reference code from search hit.
2024-01-12T20:28:25.556673836Z System.NullReferenceException: Object reference not set to an instance of an object.
2024-01-12T20:28:25.556678936Z at EPiServer.Find.Personalization.PersonalizationTrackConventions.GetRefCodeFromSearchHit(SearchHit`1 hit, String& refCode)
2024-01-12T20:28:25.556685636Z System.NullReferenceException: Object reference not set to an instance of an object.
2024-01-12T20:28:25.556692037Z at EPiServer.Find.Personalization.PersonalizationTrackConventions.GetRefCodeFromSearchHit(SearchHit`1 hit, String& refCode)
Honestly, I don't know what this could possibly be from, and I am usure when it started. Any ideas?
Since we do have a few search phrases in our dashboard, it seems that the process might be working. But new phrases aren't showing up. Online, the Autocomplete - Opti Support Doc mentions sources of suggestions, including “Automatically generated, based on previous searches, that resulted in at least one hit”. This is just not happening for us.