A/B Testing - StickySiteKpi.AddSessionOnLoadedContent throws NullReferenceException when examining referer header under specific conditions


Hello (Episerver) World!

I believe I have found a rather interesting little bug within the EPiServer.Marketing.KPI.2.4.1 package. I have searched through the release notes to see if a similar bug had been reported and fixed, but I did not find anything. I want to report what I found to see if there is a known solution for this issue and to raise awareness in case there is not a known solution. The site in question is currently running CMS 10.10.x. 

To give some context, an editor recently started an A/B test on the Start Page with a "Time on Page" KPI and a "Site Stickiness" KPI. After this was set up, we started to see an increase in NullReferenceException being reported in Application Insights for a small set of global assets, including one in particular that was going to be included in a wide-ranging marketing email. Here is the stack trace for the error in question:


at EPiServer.Marketing.KPI.Common.StickySiteKpi.AddSessionOnLoadedContent (EPiServer.Marketing.KPI, Version=, Culture=neutral, PublicKeyToken=8fe83dea738b45b7) 
at System.EventHandler`1.Invoke (mscorlib, Version=, Culture=neutral, PublicKeyToken=b77a5c561934e089) 
at EPiServer.Core.Internal.DefaultContentEvents.RaiseContentEvent (EPiServer, Version=, Culture=neutral, PublicKeyToken=8fe83dea738b45b7) 
at EPiServer.Core.Internal.DefaultContentLoader.TryGet (EPiServer, Version=, Culture=neutral, PublicKeyToken=8fe83dea738b45b7) 
at EPiServer.Core.Internal.DefaultContentLoader.Get (EPiServer, Version=, Culture=neutral, PublicKeyToken=8fe83dea738b45b7) 
at EPiServer.Web.Routing.Segments.Internal.NodeSegment.RouteDataMatch (EPiServer, Version=, Culture=neutral, PublicKeyToken=8fe83dea738b45b7) 
at System.Linq.Enumerable.All (System.Core, Version=, Culture=neutral, PublicKeyToken=b77a5c561934e089) 
at EPiServer.Web.Routing.Internal.DefaultContentRoute.RouteSegmentContext (EPiServer, Version=, Culture=neutral, PublicKeyToken=8fe83dea738b45b7) 
at EPiServer.Web.Routing.Internal.DefaultContentRoute.GetRouteData (EPiServer, Version=, Culture=neutral, PublicKeyToken=8fe83dea738b45b7) 
at System.Web.Routing.RouteCollection.GetRouteData (System.Web, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) 
at System.Web.Routing.UrlRoutingModule.PostResolveRequestCache (System.Web, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) 
at System.Web.HttpApplication+SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute (System.Web, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) 
at System.Web.HttpApplication.ExecuteStepImpl (System.Web, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) 
at System.Web.HttpApplication.ExecuteStep (System.Web, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)

After a bit of decompiling and debugging, I determined that the error seems to reliably occur when given the following:

  • The request url is for an asset under /globalassets that does not exist (eventually triggering a 404 response)
    • This is case sensitive - /GlobalAssets does not cause this issue because of the string comparison being done in EPiServer.Marketing.KPI.Common.StickySiteKpi.IsSupportingContent
  • The request url is being entered directly in the browser's address bar, as opposed to following a link.
  • httpErrors is configured as such in the web.config:
    <httpErrors errorMode="Custom" existingResponse="Replace">
          <remove statusCode="500" subStatusCode="-1" />
          <remove statusCode="404" subStatusCode="-1" />
          <error statusCode="404" prefixLanguageFilePath="" path="/404/" responseMode="ExecuteURL" />
          <error statusCode="500" prefixLanguageFilePath="" path="/error.html" responseMode="ExecuteURL" />
    • We have a dedicated content type for handling 404s that requires us to load the Start Page (ContentReference.StartPage) in order to render navigation, footer, and the like as part of a multi-site setup.
  • The visitor in question has not yet visited the corresponding Start Page since the A/B test had started.
    • This means that the request will not include the SSK_ cookie that is used to track Sticky Site KPI conversions.
    • The decompiler shows that when the cookie does not exist and we are loading the content under test (the Start Page in this case), it checks the current request url to determine if the url points to either: a png file, a css file, a global asset (/globalassets), or a content asset (/contentassets).
    • Since we did not redirect to /404/, the current url is still /globalassets/remainingpath.pdf

Given the above conditions, the Start Page will be loaded from the content repository as part of rendering the Not Found page and EPiServer.Marketing.KPI.Common.StickySiteKpi.AddSessionOnLoadedContent will subsequently be called. Because the Start Page is the content under test, and because the current url contains "/globalassets", and because the SSK_ cookie is not included in the request, EPiServer.Marketing.KPI.Common.StickySiteKpi.IsSupportingContent will return "true" and the KPI will unsuccessfully attempt to dereference HttpContext.Current.Request.UrlReferrer.AbsolutePath.

Even though this bug mainly occurs when the url will cause a 404 response, if the server error response gets cached by the CDN for an asset that will eventually be published (which happened here), this can quickly become a large issue if the PDF/asset is highly important for some business reason. I don't have a question here per se but I am interested in any comments or critiques of my analysis as well as finding out if others have run into this problem :)



Nov 02, 2019 1:33
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.