Magnus von Wachenfeldt
Feb 12, 2010
  9826
(1 votes)

Making EPiServer and Search Engines Work Together

With the dawn of search appliance everybody wants to rack up an external box in their server hall, leaving the gritty details of indexing and searching to companies who specialize in doing it best.

We often forget how picky these magic boxes are. They can not think for themselves and thrive on being provided exact directions on how to correctly index your information. That is where we more often than not fail, many times without even knowing we failed.

The Problem

When deleting or moving pages in EPiServer, there is no information stored about that specific page’s history. You either get 200 OK or 404 Not Found, taking no concern whether that page has actually been moved or even deleted. This confuses search engines, clutters your index and breaks links from external sources when restructuring your page tree.

By attaching to some of the events exposed by EPiServer’s DataFactory I will try and fix this problem with a custom module, so let’s implement IHttpModule and get running.

Wiring Up

First, let us take a look on initializing the module.

   1: public void Init(HttpApplication context)
   2: {
   3:     SetupEvents();
   4:  
   5:     context.EndRequest += new EventHandler(context_EndRequest);
   6: }

In SetupEvents() we need to listen to the relevant events.

   1: private Type _urlMapperType = typeof(SimpleSqlUrlMapper);
   2: private static readonly object _lockObject = new object();
   3: private static bool _initialized = false;
   4:  
   5: private IUrlMapper Mapper
   6: {
   7:     get
   8:     {
   9:         return Activator.CreateInstance(_urlMapperType) as IUrlMapper;
  10:     }
  11: }
  12:  
  13: private void SetupEvents()
  14: {
  15:     if (!_initialized)
  16:     {
  17:         try
  18:         {
  19:             if (Monitor.TryEnter(_lockObject))
  20:             {
  21:                 this.Mapper.GenerateSchema();
  22:  
  23:                 EPiServer.DataFactory.Instance.DeletedPage += 
  24:                     new EPiServer.PageEventHandler(Instance_DeletedPage);
  25:                 
  26:                 EPiServer.DataFactory.Instance.MovedPage += 
  27:                     new EPiServer.PageEventHandler(Instance_MovedPage);
  28:                 
  29:                 EPiServer.DataFactory.Instance.PublishedPage += 
  30:                     new EPiServer.PageEventHandler(Instance_PublishedPage);
  31:  
  32:                 _initialized = true;
  33:             }
  34:         }
  35:         finally
  36:         {
  37:             Monitor.Exit(_lockObject);
  38:         }
  39:     }
  40: }

Locking here is very important, since we do not want a possible race condition, resulting in hooking up the events multiple times. I generate a schema for the SimpleSqlUrlMapper. Then, we let a static boolean tell us whether the module has been initialized or not.

IUrlMapper is a simple interface with just a couple of methods. I was thinking about splitting responsibility for some things into another interface, but in the end I wanted something that’s just simple to set up.

   1: public interface IUrlMapper
   2: {
   3:     void GenerateSchema();
   4:     void Add301MovedPermanentlyStatus(string oldUrl, string newUrl);
   5:     void Add410GoneStatus(string url);
   6:     void RemoveMappings(string url);
   7:     bool Exists(string url);
   8:     string GetRedirectUrl(string oldUrl);
   9:     HttpStatusCode? GetStatusCode(string url);
  10: }

Now. let us see what is hidden in context_EndRequest()

   1: private void context_EndRequest(object sender, EventArgs e)
   2: {
   3:     HttpApplication app = sender as HttpApplication;
   4:     if (app != null && app.Context.Response.StatusCode == (int)HttpStatusCode.NotFound)
   5:     {
   6:         HttpStatusCode? statusCode = 
   7:             this.Mapper.GetStatusCode(app.Context.Request.Url.AbsolutePath);
   8:  
   9:         if (statusCode != null)
  10:         {
  11:             app.Context.Server.ClearError();
  12:             app.Context.Response.Clear();
  13:             app.Context.Response.StatusCode = (int)statusCode.Value;
  14:  
  15:             if (statusCode == HttpStatusCode.MovedPermanently)
  16:             {
  17:                 app.Context.Response.AddHeader("Location", 
  18:                     this.Mapper.GetRedirectUrl(app.Context.Request.Url.AbsolutePath));
  19:             }
  20:         }
  21:     }
  22: }

We need to attach to EndRequest, since the Error event will not work under IIS7 in Integrated Mode. The code is pretty self explanatory. If we get a 404, check if we have mapped this URL and respond with the correct status code.

I will not go into the implementation details of the UrlMapper, because that is not really relevant to the problem. Instead, let’s see what happens with this module set up!

Shuffling Pages Around

Let’s create a new page and fiddle around to see what happens!

new page

Now, let’s move it to “News”.

moved page once

And we end up with this in our table.

table moved once

That is all fine and dandy. Now let’s see if our magic http module works by browsing to http://acme/en/test-page/ !

redirect first page

Hooray! We got a 301 and the browser redirected us. Let us move the page around a little more to see what happens.

moved page twice

This should make both /en/test-page/ and /en/News/test-page/ tell us that this page now resides under /en/Events/test-page/.

table moved twice

…but let’s say you decided you wanted it under the start page after all.

moved page to start position

That would make us have to remove the old mapping aside from making the other locations point to the new one.

table moved page to start position

Now, let’s say you’re sick of this page and its infidelity with the page structure and you decide to remove it.

moved page to waste basket

That should teach it. Let’s see what’s in the table now.

table moved page to waste basket

Let’s open up Firefox and open one one of the URLs.

410 gone

Mission accomplished! But, what happens if we create another test-page under the root now?

new page

The mapping should be gone. Let’s look.

mapping gone

It is.

 

Let’s see if we can change the URL segment of the page too.

change url segment

Should be updated in the table now.

table change url segment

 

Disclaimer

This is the result of a late night hack, thus it is probably riddled with bugs. However, it is a good start towards building EPiServer sites with better support for search engines. There are a few known bugs or things I just neglected for now due to sleep deprivation, such as:

* Weird behaviour when moving pages out of waste basket
* No support for mapping descendants of moved pages yet

Go ahead and try it out, and please give me a shout if you make any improvements or just want to toss ideas at me.

Install the module by putting it in your bin folder and add this to your modules section in web.config:
<add name="StatusCodeModule" type="Nansen.StatusCodes.Modules.StatusCodeModule, Nansen.StatusCodes" />

Feb 12, 2010

Comments

Sep 21, 2010 10:33 AM

Awesome!

Sep 21, 2010 10:33 AM

Combining the logic above with the 404 Handler from EPiCode on CodeResort would be a great idea. (https://www.coderesort.com/p/epicode/wiki/404Handler).

Sep 21, 2010 10:33 AM

I like the idea. I'm definitely making a module based on this. Nice work!
I see you have switched company since I saw you last as well, good luck you crazy bastard. :)

/ Daniel Ovaska

Sep 21, 2010 10:33 AM

Hey there. :) Thank you for the kind words!

There is an update at http://world.episerver.com/Blogs/Magnus-von-Wachenfeldt/Dates/2010/2/Update-to-the-Url-Mapping-Module/ with bugfixes and stuff. It's going to be used on a very big site, so I'll ask them for updates when they're done...

Please login to comment.
Latest blogs
Plug-in manager is back in CMS 12

Plug-in manager is back in the UI, what is it and how can i use it?

Luc Gosso (MVP) | Oct 6, 2022 | Syndicated blog

Display Child Pages in Content Delivery API Response

The below example will implement an instance of IContentConverterProvider to customise the serialisation of PageData and output child pages in the...

Minesh Shah (Netcel) | Oct 4, 2022

Bring the Report Center back in Optimizely CMS 12

The Report Center has been a part of Optimizely CMS since its first debut in version 5R2 in 2008, but in CMS 12, it's removed! Don't despair! Make...

Tomas Hensrud Gulla | Oct 4, 2022 | Syndicated blog

Customizing Property Lists in Optimizely CMS

Generic property lists is a cool editorial feature that has gained a lot of popularity - in spite of still being unsupported (officially). But if y...

Allan Thraen | Oct 2, 2022 | Syndicated blog

Optimizely names Luminary Senior Developer, Ynze Nunnink, OMVP

Luminary Senior Developer and Optimizely Lead, Ynze Nunnink has secured the coveted position of Optimizely MVP. Earning a Platinum badge for his...

Ynze | Oct 2, 2022 | Syndicated blog

Content Delivery API – The Case of the Duplicate API Refresh Token

Creating a custom refresh provider to resolve the issues with duplicate tokens in the DXC The post Content Delivery API – The Case of the Duplicate...

David Lewis | Sep 29, 2022 | Syndicated blog