Blog posts by Paul Gruffydd

Developer meetup - Manchester, 23rd January

Thu, 09 Jan 2025 12:45:40 GMT

Yes, it's that time of year again where tradition dictates that people reflect on the year gone by and brace themselves for the year ahead, and what better way to kick start your year than an evening of talks, food, drink and of course plenty of time to mingle with other members of our fantastic Optimizely community. Our first Optimizely developer meetup of 2025 is just 2 weeks away on the evening of Thursday the 23rd of January and we can't wait to welcome you. We've got a fantastic schedule of talks lined up from some great speakers all in the luxurious surroundings of Manchester's Everyman cinema.

We're just putting the finishing touches to the running order but the lineup includes OMVP (and UK dev meetup legend) Mark Everard of First Three Things talking about how you can get the most out of the new Visual Builder, and the ever entertaining and informative Simon Chapman of Optimizely giving an update on all that's new and upcoming in the world of Optimizely.

⚠️UPDATE⚠️

We have now finalised the running order and I'm pleased to say we've got 4 fantastic talks lined up:

Mark Everard (First Three Things) - Making the Most of Visual Builder
Anil Patel (Knight Frank) - Azure Business Communication Services integration with Optimizely Forms
Andy Blyth (26) - The fragment conundrum - Overcoming Fragment Limits in GraphQL
Simon Chapman (Optimizely) - Optimizely Roadmap Update

As always, tickets are free (though spaces are limited so please register) and we'll be laying on food and drinks during the evening.

Doors open from 17:15 and talks begin at 18:00. All the talks should be wrapped up by 20:30 but please do stick around afterwards in the bar area for a drink and a chat.

This promises to be a fantastic event so register now to guarantee your place via the form on the Optimizely events page (select "Manchester UK - Jan 23"):
https://www.optimizely.com/local-dev-meetups

Venue details

Everyman Manchester St. John's
ABC Buildings
23 Quay Street
Manchester
M3 4AS

🌐 https://www.everymancinema.com/manchester-st-johns#venueDetails

🗺️ View on Google maps

See you there!

Developer Meetup - Manchester, 1st March

Mon, 27 Feb 2023 12:28:16 GMT

This Wednesday (the 1st of March) sees the first UK developer meetup of 2023 and our first in Manchester for far too long. We've got some brilliant talks lined up from some great speakers in a fantastic venue (The Everyman Cinema, Manchester St. John's).

The lineup includes:

Mark Stott, Lead Optimizely Developer at 26 talking about simplifying a new website build on CMS 12
Mark Everard, Optimizely Consultant talking about Liquid Templating
Paul Gruffydd, Technical Director at Kin + Carta talking about GraphQL and Optimizely Content Graph
Simon Chapman, Lead Solution Architect at Optimizely talking about how to run successful Experimentation Projects with Optimizely

As always, tickets are free (though please register) and we'll be laying on food and drinks during the evening.

Doors open from 17:15 and talks begin at 18:00. All the talks should be wrapped up by 20:30 but feel free to stick around afterwards in the bar area for a drink and a chat.

This promises to be a fantastic event so be sure to register your place via the form on the Optimizely events page (select "Manchester - Mar 1"):
https://www.optimizely.com/local-dev-meetups/

Venue details

Everyman Manchester St. John's
ABC Buildings
23 Quay Street
Manchester
M3 4AS

🌐 https://www.everymancinema.com/manchester-st-johns#venueDetails

🗺️ View on Google maps

See you there!

Triggering Webhooks from custom events

Thu, 10 Jun 2021 10:36:41 GMT

In my previous post I introduced the Kin and Carta Webhooks add-on and showed how it can be used to fire webhooks based on one of 5 out-of-the-box content events. But what if you want to trigger a webhook based on something else? While those 5 events cover the more common scenarios, there are plenty of situations which they don’t cover so in this post I’m going to show how you can register your own event types which can be used to set up webhooks in the add-on UI.

The basics

Setting up your own events is pretty straightforward and follows the same process regardless of what triggers your event. The basic steps are:

Create an initializable module with a module dependency of "KinAndCarta.Connect.Webhooks.Setup.InitialiseWebhooks"
Create a new EventType for your event (in the "KinAndCarta.Connect.Webhooks.Models" namespace)
Register your event type by calling the "RegisterEventType" method of IWebhookRepository
Add a translation to give your event a friendly name
When your event occurs, trigger the webhooks using the "TriggerWebhooks" extension method

While the steps above may tell you what you need to do to get up and running with custom events, I always find it easier to look at an example rather than a list and, in this instance, instead of coming out with some examples myself, I thought I’d take inspiration from the comments on my previous post.

Creating a “First Publish” event

In response to the previous article, Stein-Viggo Grenersen mentioned using the add-on to post to social media when new articles are published. That scenario is a great example of how the add-on can be used and it’s really simple to set up using integration platforms like IFTTT, Zapier and Azure Logic Apps but, using the OOTB content publish event, there’s a risk that you’d post to social media every time the content is published in any language, even if you’re just fixing a typo. That could mean that you’re posting the same article again and again. That may work in a demo scenario but, in the real world you probably just want to post it once, the first time you publish or, for multilingual scenarios, maybe once per language. So let’s take a look at how custom events can help.

For this example, let’s create an EventType called “First publish in language” which will only fire the first time an article is published in a given language. An EventType in this context just requires a key (to identify the event) and a flag to indicate whether the event that has occurred impacts the descendants of the content item which triggered the event (for example, the Delete event sets ImpactsDescendants to true as all descendants will be deleted too). To trigger our event, we’ll need to hook in to the publishing and published events from IContentEvents. In the publishing event, we’ll check whether any published versions exist in the current language and pass that information through to the published event using the ContentEventArgs.Items Dictionary which is shared between events triggered by the same action. If no existing published versions exist, we then trigger our webhook from within the “published” event. To trigger the webhooks attached to a content item, we use the aptly named "TriggerWebhooks" extension method (from KinAndCarta.Connect.Webhooks.Extensions) and pass in our event object, the IWebhookRepository instance and any additional data we want to include about our event which, in this case, is null.

The code looks like this…

[InitializableModule]
[ModuleDependency(typeof(KinAndCarta.Connect.Webhooks.Setup.InitialiseWebhooks))]
public class FirstPublishWebhookEventsInitialisation : IInitializableModule
{
    private IWebhookRepository _webhookRepo;
    private IContentLoader _contentLoader;
    private IContentEvents _contentEvents;
    private EventType _customEvent;
    public void Initialize(InitializationEngine context)
    {
        //Get dependencies
        _contentEvents = context.Locate.Advanced.GetInstance<IContentEvents>();
        _webhookRepo = context.Locate.Advanced.GetInstance<IWebhookRepository>();
        _contentLoader = context.Locate.Advanced.GetInstance<IContentLoader>();

        //Create & register our event
        _customEvent = new EventType { ImpactsDescendants = false, Key = "FIRST_PUBLISH_IN_LANGUAGE" };
        _webhookRepo.RegisterEventType(_customEvent);

        //Setup Event handlers
        _contentEvents.PublishingContent += ContentPublishing;
        _contentEvents.PublishedContent += ContentPublished;
    }


    private void ContentPublishing(object sender, ContentEventArgs e)
    {
        var versionRepo = ServiceLocator.Current.GetInstance<IContentVersionRepository>();
        bool firstPublish;

        //Check whether a published version already exists
        //If content can be translated, check if a published version exists in the current language
        if (e.Content is ILocalizable localContent)
        {
            firstPublish = versionRepo.LoadPublished(e.ContentLink.ToReferenceWithoutVersion(), localContent.Language.Name) == null;
        }
        else
        {
            firstPublish = versionRepo.LoadPublished(e.ContentLink.ToReferenceWithoutVersion()) == null;
        }

        //Set a value for the subsequent publish event to check
        e.Items.Add("FirstPublish", firstPublish);
    }
    private void ContentPublished(object sender, ContentEventArgs e)
    {
        //Check for our FirstPublish valie
        if (e.Items["FirstPublish"] as bool? ?? false)
        {
            //TriggerWebhooks extension uses KinAndCarta.Connect.Webhooks.Extensions namespace
            e.Content.TriggerWebhooks(_customEvent, _webhookRepo, null);
        }
    }

    public void Uninitialize(InitializationEngine context)
    {
        _contentEvents.PublishingContent -= ContentPublishing;
        _contentEvents.PublishedContent -= ContentPublished;
    }
}

If we compile and run our code then look at a webhook in the Webhooks UI, here’s what we’ll see.

It’s functional, but not the most user friendly so let’s set up a translation for our event name. Translations of the event names are handled like any other translation in the CMS as you can see in the XML snippet below.

<?xml version="1.0" encoding="utf-8" ?>
<languages>
  <language name="English" id="en">
    <Webhooks>
      <EventTypes>
        <PUBLISH>Content Published</PUBLISH>
        <MOVING>Content Moving</MOVING>
        <MOVE>Content Moved</MOVE>
        <RECYCLE>Content Sent to Recycle Bin</RECYCLE>
        <DELETE>Content Deleted from Recycle Bin</DELETE>
        <FIRST_PUBLISH_IN_LANGUAGE>First publish in language</FIRST_PUBLISH_IN_LANGUAGE>
      </EventTypes>
    </Webhooks>
  </language>
</languages>

Reloading the screen, we can now see our friendly event name in the list of events and use it to fire a webhook the first time a content item is published in a given language.

Reporting on Scheduled Job execution

In another comment on the original article, Jay Wilkinson suggested using webhooks to act on the outcome of a scheduled job execution, for example, sending an email if a job fails. As you’ve probably realised, we’re not restricted to just firing webhooks on content events. We can trigger webhooks from anywhere in our code, in response to any action, however, as seen in the example above, we need to trigger the webhook using a content item which will be passed in the payload of the webhook. In this particular example though, we’re not interested in the content so we’ll trigger the webhook using the global block folder, knowing it’s always there, easily referenced and contains minimal data of its own (it's not ILocalizable so won't contain language information unlike the content root node). We are, however, interested in information about the scheduled job execution and we can add that data to the webhook payload pretty easily using the “extraData” parameter when we call TriggerWebhooks. The extraData parameter is a Dictionary<string, object> so we can easily add pretty much any data we need.

In this example we’re going to hook into the Executed event exposed by IScheduledJobEvents which fires when a scheduled job has finished executing. The basics are the same as the example above but this time we’re going to extract some information on the job which has executed and pass it into the webhook payload via the extraData parameter. Putting it all together, here’s how it looks.

[InitializableModule]
[ModuleDependency(typeof(KinAndCarta.Connect.Webhooks.Setup.InitialiseWebhooks))]
public class ScheduledJobCompletionWebhookInitialisation : IInitializableModule
{
    private IScheduledJobEvents _scheduledJobEvents;
    private IWebhookRepository _webhookRepo;
    private IContentLoader _contentLoader;
    private EventType _customEvent;

    public void Initialize(InitializationEngine context)
    {
        //Get dependencies
        _webhookRepo = context.Locate.Advanced.GetInstance<IWebhookRepository>();
        _contentLoader = context.Locate.Advanced.GetInstance<IContentLoader>();
        _scheduledJobEvents = context.Locate.Advanced.GetInstance<IScheduledJobEvents>();

        //Create & register our event
        _customEvent = new EventType { ImpactsDescendants = false, Key = "SCHEDULED_JOB_EXECUTED" };
        _webhookRepo.RegisterEventType(_customEvent);

        //Attach to scheduled job executed event
        _scheduledJobEvents.Executed += executedScheduledJob;
    }

    private void executedScheduledJob(object sender, ScheduledJobEventArgs e)
    {
        //Build extra info to pass in payload
        var jobInfo = new Dictionary<string, object>();
        jobInfo.Add("Name", e.Job.Name);
        jobInfo.Add("Success", !e.Job.HasLastExecutionFailed);
        jobInfo.Add("Message", e.Job.LastExecutionMessage);

        //Trigger webhook on asset root page
        _contentLoader.Get<IContent>(ContentReference.GlobalBlockFolder).TriggerWebhooks(_customEvent, _webhookRepo, jobInfo);
    }

    public void Uninitialize(InitializationEngine context)
    {
        //Clean up event
        _scheduledJobEvents.Executed -= executedScheduledJob;
    }
}

As before, you’ll probably want to add a translation to give the event a friendly name.

Once you’ve compiled and run the solution you can set up a webhook which fires on scheduled job executions, just remember to select the root node or the global blocks folder when setting up the webhook as that’s the node we’re attaching our custom event to. The payload data will look something like this.

{
  "ReferencedBy": [],
  "Content": {
    "ContentLink": {
      "Id": 3,
      "WorkId": 0,
      "GuidValue": "e56f85d0-e833-4e02-976a-2d11fe4d598c",
      "ProviderName": null,
      "Url": null,
      "Expanded": null
    },
    "Name": "For All Sites",
    "Language": null,
    "ExistingLanguages": [],
    "MasterLanguage": null,
    "ContentType": [
      "Folder",
      "SysContentFolder"
    ],
    "ParentLink": {
      "Id": 1,
      "WorkId": 0,
      "GuidValue": "43f936c9-9b23-4ea3-97b2-61c538ad07c9",
      "ProviderName": null,
      "Url": null,
      "Expanded": null
    },
    "RouteSegment": "SysGlobalAssets",
    "Url": null,
    "Changed": "1999-01-01T00:00:00Z",
    "Created": "1999-01-01T00:00:00Z",
    "StartPublish": null,
    "StopPublish": null,
    "Saved": "1999-01-01T00:00:00Z",
    "Status": null
  },
  "ExtraData": {
    "Name": "Publish Delayed Content Versions",
    "Success": true,
    "Message": "Nothing was published."
  },
  "ContentInfo": {
    "ContentId": 3,
    "ContentGuid": "e56f85d0-e833-4e02-976a-2d11fe4d598c",
    "ContentLanguage": "",
    "Name": "For All Sites",
    "Url": null
  },
  "EventType": "SCHEDULED_JOB_EXECUTED",
  "EventTime": "2021-06-10T10:00:10.7754622Z"
}

We can then easily set up a receiver in, for example, Azure Logic Apps to receive our webhook and use the data passed in the extraData parameter to perform actions such as sending a Slack message if a specific job fails.

So there we have it - just a couple of simple examples to show how you can use Webhooks with custom events but hopefully it’s conveyed how simple and flexible it is to set up your own event types. I look forward to hearing how you put it to use in your own projects.

As I mentioned in the previous post, the add-on is available in the Episerver NuGet feed under “KinAndCarta.Webhooks” and requires no particular set-up other than installing the package. Full source code is also available on GitHub under an MIT license.

Using content webhooks with Episerver

Wed, 31 Mar 2021 09:30:07 GMT

The concept of webhooks isn't a new one but in recent years it has become a core component to many headless and SaaS implementations. Without direct access to the underlying data, solutions built in this way typically rely on HTTP APIs to access data and expose functionality, but relying on HTTP APIs alone can result in unnecessary overheads. Excessive data transfer and the network latency introduced can, if not carefully managed, make applications feel clunky and unresponsive. That’s where webhooks come in.

At a high level, webhooks offer a basic pub/sub pattern implemented using standard HTTP requests. A URL is registered with the source system and when certain events occur in that system, an HTTP request is made to the URL containing information on the event which occurred. The data sent in the http request can then be processed by the receiving system, allowing for scenarios where data is refreshed in response to changes to that data rather than having to periodically check.

While headless content delivery may be a key use case for web hooks, it’s certainly not the only use case. Webhooks can be of use in a whole range of scenarios, particularly when combined with low-code integration platforms such as Azure Logic Apps, IFTTT and Zapier. These platforms provide ready built integrations to an extensive catalogue of services which you can put together like lego blocks enabling you to build complex integrations and data processing tasks without the need to write or deploy any code, and all of them can be triggered by webhooks.

As great as this may sound, there's something missing. While Episerver / Optimizely does have some webhook functionality (most notably in Forms though also in Campaign), there is no functionality to trigger webhooks from content events. I can understand why this isn’t provided out-of-the-box, after all, we can programmatically attach handlers to any Episerver events and perform whatever actions we need. But codebase updates and deployments introduce their own inefficiencies and, with decoupled architectures becoming more common, webhooks offer a simpler, more unified solution. Sounds like a job for an add-on...

Introducing Kin and Carta Webhooks

At February’s Developer Happy Hour I demonstrated an add-on I’ve been working on which adds webhook functionality to Episerver / Optimizely CMS solutions and I’m pleased to announce that it is now available via the Episerver NuGet feed. After installing the add-on, an additional “webhooks” menu item is added to the CMS navigation for users with admin access which links to the webhook UI. The UI allows for webhooks to be created, tested, updated and deleted as well as allowing for an at-a-glance view of the last result for each webhook. Clicking on the last result pops up the full response received, allowing for easier debugging.

Creating a webhook

To set up a new webhook, click the “New Webhook” button and enter a name to identify the webhook and a URL which will be called when the webhook is fired, then select a parent node which can be used to limit triggering the webhook to only the selected node or its children.

You should also select one or more content events which will determine when the web hook is triggered. Out of the box, the add on supports 5 content events:

Publish - Called once a content item has been published
Moving - Called before a content item has been moved
Move - Called after a content item has been moved
Recycled - Called after a content item has been moved to the recycle bin
Deleted - Called before a content item has been permanently deleted

N.B. Content save events have been intentionally ignored as they occur almost constantly while editing content however it is possible to register your own events if this is required. I’ll cover how to do that in a subsequent post.

You have the option to select one or more content types so that the webhook will trigger only on events involving content of the selected types. Leaving this blank defaults to triggering on all content types (assuming they’re in the correct part of the tree). Should you require, you can also add up to 5 custom HTTP headers which could be used for sending authentication information, keys or any other additional information your receiver may require in the header.

Once all of the relevant information has been entered, click save to save the webhook and return to the webhook overview screen. You can now click on the “Test” button to trigger the webhook as though a content event had occurred. To do this, the add-on finds the first content item which matches the criteria set and triggers the webhook using that content item. The result of the test will appear in a popup.

Payload data

When one of the supported events is triggered by a content event, any webhooks targeted to that event, content location and content type will be retrieved and a POST request will be made to the defined URL. This request is made on a background thread to avoid impacting the default processing of the action which triggered the event. The body of the post request contains information about the event fired, the content that triggered it and any other content items impacted by the event in JSON format. The top level JSON properties are:

Content - a JSON representation of the content which triggered the event. This uses the content delivery API to serialise the content and so the structure of the serialised content data should match data retrieved from the API, including customisations such as flattening the structure.
ContentInfo - high level information about the content (Name, IDs, Language, URL)
ReferencedBy - information about content items (Name, IDs, Language, URL) which reference the content which triggered the event. This can be useful when using webhooks to invalidate remote caches.
EventType - the name of the event which occurred
EventTime - the date/time at which the event occurred

There is also a field called ExtraData, though this exists for extension purposes and so will currently be null. More on this in a future post.

An easy way to explore the structure of the payload is to set up a webhook pointed to a service like webhook.site. To try this out, simply visit https://webhook.site/ and you will be assigned a temporary URL. You can then set up a webhook in Episerver using that URL and send a test. Any requests sent to that URL will appear as they are received and can be inspected to view the payload and header data.

Token Replacement

When creating a webhook, tokens can be used within the URL and custom header fields to allow information such as keys or environment-specific values to be pulled in. Tokens are in the format ${TokenName} where TokenName is a key registered using the RegisterPlaceholder(string token, string value) method of the IWebhookRepository. The value associated with the token will replace the token placeholder. In the event that the token name doesn't match a key, the token placeholder will be left as-is.

Registering tokens and their values will typically be done in an IInitializableModule though, if you want to register all appsettings values from your web.config file, you can add the following app setting to your config:
<add key="WH:AutoRegisterAppSettings" value="true" />

N.B. Before using this config value, ensure you are aware of the implications. By registering all appsettings like this, it is possible to extract potentially sensitive data from configuration and send it to an arbitrary URL.

Further information

During the February Dev Happy Hour I demonstrated a few of the ways the add-on could be used, including a demo of how we can use webhooks with Azure Logic Apps to selectively invalidate a CDN cache when cached content is updated. You can find the recording on the EMEA Dev Happy Hour page (the webhooks bit starts at the 30 minute mark).

The add-on is available in the Episerver NuGet feed under “KinAndCarta.Webhooks” and requires no particular set-up other than installing the package. Full source code is also available on GitHub under an MIT license.

I hope you find this add-on useful and do let me know in the comments if you’ve got any feedback, questions, comments or suggestions.

Using KQL to list popular content from Profile Store

Wed, 04 Dec 2019 12:31:57 GMT

Shortly after Profile Store was released I wrote a post about how we can use it to do more than just process customer-centric data for display in insight and, instead, look at more event-centric data like popular site content. While the solution did the job, there was plenty of room for improvement. As I said in the post, “In an ideal world, we could do the aggregation as part of our query but unfortunately this isn’t supported right now”. Well, following the recent announcements about KQL support, this should now be possible so I thought it would be a good time to revisit that post and see how we could implement this “popular content” functionality today.

Introducing KQL

For those not familiar with KQL, it stands for Kusto Query Language and you’re most likely to have seen (or even used) it when interrogating data in Application Insights in Azure. There’s a brief overview of using KQL with profile store in the Profile Store documentation and a bit more detail in this post from Dmytro Duk. I won’t cover-off the specifics of KQL here as there are other resources out there for that purpose but it’s worth noting that it’s a huge step up from the previous filter-based queries we could perform against Profile Store data. Where previously we could only filter and sort the results returned, KQL gives us the ability to project, aggregate, join and transform that data. You can even apply machine learning algorithms for clustering data if you’re that way inclined, but for our purposes, we’ll stick with the basics.

Tracking

For the reasons mentioned in the original post, we’ll track our page views using the PageViewTrackingAttribute in EPiServer.Tracking.PageView which records epiPageView events in Profile Store which look like this:

{
    "TrackId": null,
    "DeviceId": "91c08036-5335-4762-9660-d17c5677fba9",
    "EventType": "epiPageView",
    "EventTime": "2019-11-29T13:57:13.9670478Z",
    "Value": "Viewed Start",
    "Scope": "463470c3-3eca-41d3-8b12-3f7f92f62d34",
    "CountryCode": "Localhost",
    "PageUri": "http://localhost:59422/",
    "PageTitle": null,
    "RemoteAddress": "::1",
    "Payload": {
        "epi": {
            "contentGuid": "bd437cef-41bd-4ebc-8805-0c20fcf4edcf",
            "language": "en",
            "siteId": "463470c3-3eca-41d3-8b12-3f7f92f62d34",
            "ancestors": [
                "43f936c9-9b23-4ea3-97b2-61c538ad07c9"
            ],
            "recommendationClick": null
        }
    },
    "User": {
        "Name": "admin",
        "Email": "email@address.blah",
        "Info": {}
    },
    "SessionId": "a7751741-6936-4f89-81ac-e50dd980ab13"
}

We can search on any of the properties shown in the JSON structure above but there’s a problem. In the original post we had the ability to filter by content type (e.g. return a list of popular content of type ArticlePage) but content type isn’t recorded in the epiPageView event so how did that work? Well, in the original post, the data was processed in a scheduled job (as it would have been too slow and resource-intensive to do on-the-fly) and stored in the dynamic data store so, at that point we could augment the data with, for example, the page type. Our searching was then done against the DDS rather than Profile Store.

In this instance I want to avoid using scheduled jobs and DDS queries if at all possible and try to condense everything we need into a KQL query, so we’re going to have to store the content type as part of that tracking request. We can do that by creating a tracking data interceptor which allows us to modify the page tracking data before it’s sent to Profile Store. This involves creating a class which implements ITrackingDataInterceptor, registering it as an instance of ITrackingDataInterceptor and swapping out the standard payload for one which contains an additional field called typeId which will hold our content type. This is done in the intercept method like this:

[ServiceConfiguration(ServiceType = typeof(ITrackingDataInterceptor), Lifecycle = ServiceInstanceScope.Singleton)]
public class ContentTypeTrackingInterceptor : ITrackingDataInterceptor
{
    private IContentLoader _contentLoader;
    private IContentTypeRepository _contentTypeRepository;

    public int SortOrder => 100;

    public ContentTypeTrackingInterceptor(IContentLoader contentLoader, IContentTypeRepository contentTypeRepository)
    {
        _contentLoader = contentLoader;
        _contentTypeRepository = contentTypeRepository;
    }

    public void Intercept<TPayload>(TrackingData<TPayload> trackingData)
    {
        if (trackingData == null || trackingData.Payload == null)
        {
            return;
        }
        if (!(trackingData.Payload is EPiServer.Tracking.PageView.EpiPageViewWrapper payload))
        {
            return;
        }

        // Create replacement Epi payload object
        var pageView = new EpiPageViewWithType(payload.Epi);
        var page = _contentLoader.Get<IContent>(payload.Epi.ContentGuid);
        pageView.TypeId = page.ContentTypeID;
        payload.Epi = pageView;
    }
}

If we run the site now and click around a bit, the recorded events should now have our Payload.epi.typeId field like this:

{
    "TrackId": null,
    "DeviceId": "91c08036-5335-4762-9660-d17c5677fba9",
    "EventType": "epiPageView",
    "EventTime": "2019-11-29T13:57:13.9670478Z",
    "Value": "Viewed Start",
    "Scope": "463470c3-3eca-41d3-8b12-3f7f92f62d34",
    "CountryCode": "Localhost",
    "PageUri": "http://localhost:59422/",
    "PageTitle": null,
    "RemoteAddress": "::1",
    "Payload": {
        "epi": {
            "typeId": 23,
            "contentGuid": "bd437cef-41bd-4ebc-8805-0c20fcf4edcf",
            "language": "en",
            "siteId": "463470c3-3eca-41d3-8b12-3f7f92f62d34",
            "ancestors": [
                "43f936c9-9b23-4ea3-97b2-61c538ad07c9"
            ],
            "recommendationClick": null
        }
    },
    "User": {
        "Name": "admin",
        "Email": "email@address.blah",
        "Info": {}
    },
    "SessionId": "a7751741-6936-4f89-81ac-e50dd980ab13"
}

KQL query

So now we’ve got the data in the correct format, let’s look at how we can query that data to retrieve popular content. To run a KQL query against Profile Store, first we need to wrap our query in a query object containing the KQL query and the scope to run it against like this:

{ 
    "Query": "...", 
    "Scope": "..." 
}

N.B. Unless you’ve explicitly set it to something else, the value of ‘Scope’ will be the GUID of the site you’re tracking which can be accessed as SiteDefinition.Current.Id.

This object needs to be POSTed as the body of a request to:
https://[your profilestore host name here]/api/v2.0/TrackEvents/preview

You’ll also need to set the following headers:
Authorization: epi-single [your token here]
Content-Type: application/json-patch+json

In our original post we started by querying all epiPageView events in the last 24 hours which we could do quite easily with a query like this:

Events
| where EventTime between (ago(24h) .. now())
and EventType == 'epiPageView'

If we use that query though, we’d still need to manually aggregate the data in a scheduled job as we did before, so we need to be more selective in what we return and we need to retrieve aggregated data rather than a raw listing of events. To do that we can add some additional predicates to the where clause then use KQL’s 'summarize' operator alongside the 'top' operator to return us the guids for the top n content items of a given type, in a given language, under a given ancestor, ordered by number of views. Putting that together, we get a query like this:

Events
| where EventTime between (ago(24h) .. now())
    and EventType == 'epiPageView'
    and Payload.epi.language == 'en'
    and Payload.epi.ancestors contains('bd437cef-41bd-4ebc-8805-0c20fcf4edcf')
    and Payload.epi.typeId == 15
| summarize Count = count() by Payload.epi.contentGuid
| top 5 by Count desc

But there’s a caveat - data returned from a KQL query to Profile Store is returned in a fixed structure (the epiPageView JSON structure shown above) and when we call summarize, this cuts down the data returned to just include the count (if we assign it a name) and the values used to group the data. In our case we’re grouping the data by the page guid (Payload.epi.contentGuid) but, as that’s not a top-level variable name, we can’t retrieve it and get a 500 error. The workaround is to map the variable to one of the top-level property names. In my case I’ve chosen to use ‘Value’ so our slightly tweaked KQL query looks like this:

Events
| where EventTime between (ago(24h) .. now())
    and EventType == 'epiPageView'
    and Payload.epi.language == 'en'
    and Payload.epi.ancestors contains('bd437cef-41bd-4ebc-8805-0c20fcf4edcf')
    and Payload.epi.typeId == 15
| summarize Count = count() by Value = tostring(Payload.epi.contentGuid)
| top 5 by Count desc

Which will return us a list of content GUIDs as shown below which can then be used to load in the content items associated with each GUID.

{
    "items": [
        {
            "Value": "74964f63-98c9-4d05-8069-5f8221e0c6ad"
        },
        {
            "Value": "59f81174-6502-4c18-9e71-19cd7a6f2980"
        },
        {
            "Value": "6f3891ee-c7a1-4fc5-a12c-95222f05b537"
        },
        {
            "Value": "5f280886-1e3a-4ae0-a283-6b7c180abc82"
        },
        {
            "Value": "e99088f3-2394-49c0-8205-2e3a1aecc8f7"
        }
    ],
    "count": 5
}

Putting it all together

All that remains is to wrap that all in some code to substitute in the appropriate values into the query, make the requests and process the response, giving us two very similar methods to the “GetPopularPages” methods in the previous post but with one important difference. As we’re making the queries in real-time, we don’t have to have a fixed window for what’s classed as recent so, if articles attract thousands of views an hour but FAQs only attract a few per day, we could set the window for a recent view of an article to be 6 hours but an FAQ to be 7 days vastly improving the reusability of our block type.

Putting it all together, our code looks like this:

public class ProfileStoreHelper
{
    //Settings
    private string _apiRootUrl = ConfigurationManager.AppSettings["episerver:profiles.ProfileApiBaseUrl"];
    private string _appKey = ConfigurationManager.AppSettings["episerver:profiles.ProfileApiSubscriptionKey"];
    private string _eventUrl = "/api/v2.0/TrackEvents/preview";
    private string _scope = ConfigurationManager.AppSettings["episerver:profiles.Scope"] ?? SiteDefinition.Current.Id.ToString();

    private IContentLoader _contentLoader;
    private IContentTypeRepository _contentTypeRepository;

    public ProfileStoreHelper(IContentTypeRepository contentTypeRepository = null, IContentLoader contentLoader = null)
    {
        _contentTypeRepository = contentTypeRepository ?? ServiceLocator.Current.GetInstance<IContentTypeRepository>();
        _contentLoader = contentLoader ?? ServiceLocator.Current.GetInstance<IContentLoader>();
    }

    /// <summary>
    /// Get pages of a given type
    /// </summary>
    public IEnumerable<T> GetPopularPages<T>(ContentReference ancestor, string lang, int resultCount = 5, int recentHours = 24) where T : PageData
    {
        var contentTypeId = _contentTypeRepository.Load<T>().ID;
        var ancestorGuid = _contentLoader.Get<IContent>(ancestor).ContentGuid;
        var hits = GetRecentContentResponse(ancestorGuid, lang, resultCount, recentHours, contentTypeId);
        return hits?.Items?.Select(x => _contentLoader.Get<T>(x.Value)) ?? Enumerable.Empty<T>();
    }

    /// <summary>
    /// Get all popular content regardless of type
    /// </summary>
    public IEnumerable<IContent> GetPopularPages(ContentReference ancestor, string lang, int resultCount = 5, int recentHours = 24)
    {
        var ancestorGuid = _contentLoader.Get<IContent>(ancestor).ContentGuid;
        var hits = GetRecentContentResponse(ancestorGuid, lang, resultCount, recentHours);
        return hits?.Items?.Select(x => _contentLoader.Get<IContent>(x.Value)) ?? Enumerable.Empty<IContent>();
    }

    /// <summary>
    /// Make request to profile store API
    /// </summary>
    private RecentContentResponse GetRecentContentResponse(Guid ancestorGuid, string lang, int resultCount = 5, int recentHours = 24, int typeId = 0)
    {
        var requestBody = $"{{\"Query\": \"{GenerateKQLQuery(ancestorGuid, lang, resultCount, recentHours, typeId)}\", \"Scope\": \"{_scope}\" }}";
        var req = new RestRequest(_eventUrl, Method.POST);
        req.AddHeader("Authorization", $"epi-single {_appKey}");
        req.AddParameter("application/json-patch+json", requestBody, ParameterType.RequestBody);
        req.RequestFormat = DataFormat.Json;
        req.AddBody(requestBody);
        var client = new RestClient(_apiRootUrl);
        var getEventResponse = client.Execute(req);
        return JsonConvert.DeserializeObject<RecentContentResponse>(getEventResponse.Content);
    }

    /// <summary>
    /// Construct KQL query
    /// </summary>
    private string GenerateKQLQuery(Guid ancestorGuid, string lang, int resultCount = 5, int recentHours = 24, int typeId = 0)
    {
        var kqlQueryObj = @"Events 
            | where EventTime between(ago({0}h) .. now()) 
                and EventType == 'epiPageView' 
                and Payload.epi.language == '{1}' 
                and Payload.epi.ancestors contains('{2}') 
                {3}
            | summarize Count = count() by Value = tostring(Payload.epi.contentGuid)
            | top {4} by Count desc";
        //Only add type restriction if a type has been specified
        var typeQuery = typeId > 0 ? $"and Payload.epi.typeId == {typeId}" : string.Empty;
        return string.Format(kqlQueryObj, recentHours, lang, ancestorGuid.ToString(), typeQuery, resultCount);
    }
}

public class RecentContentResponse
{
    public int Count { get; set; }
    public RecentContentResponseItem[] Items { get; set; }
}

public class RecentContentResponseItem
{
    public Guid Value { get; set; }
}

Which we can then call from a block controller to give us something like this

As you can see, by taking advantage of the recent API updates in Profile Store we’ve managed to remove our reliance on scheduled jobs and the DDS, and cut down the amount of code we need to get useful data while improving the relevance of the results we return.

As with the previous post, I’ve added the code (including the block) to a Gist on GitHub but do bear in mind that this has been created as a proof-of-concept rather than a battle-hardened, production-ready feature so use it with caution.

Templating emails from Episerver Forms

Thu, 08 Aug 2019 12:59:26 GMT

This may sound obvious but maintaining brand consistency across channels is pretty important yet, all too often, transactional emails seem to fall through the cracks. Am I alone in feeling slightly disappointed when I fill in a form on a website only to receive an unbranded, generic, text-only “thanks for filling in our form” email which could have come from anyone?

A fairly comprehensive solution to this problem (and several others) is to invest in marketing automation technology such as Episerver Campaign and an increasing number of companies are going down that route but there are still many situations where that kind of approach is overkill. In some circumstances it’s sufficient to just use Episerver Forms, particularly given that, should your requirements grow to include marketing automation, you can extend the functionality of Forms using one of the ready-built connectors.

A big advantage of Episerver Forms over its predecessor (XForms 😱) is that Forms includes a mechanism out of the box to allow you to send different emails with editor controllable content to multiple email addresses (including the person filling in the form), functionality which we used to have to write ourselves in XForms. The downside of these emails is that you only get a WYSIWYG editor to manage the full content of the email which makes it difficult to produce fully branded emails and, as each email is created individually, this increases the burden on editors when it comes to managing essential shared text such as disclaimers which are required across all emails. If we could apply a template to those emails, wrapping the editable content, then we alleviate both of those issues.

Within Forms, the processing of submitted form data is handled using a concept called actors and, as developers, we’re free to create as many actors as we like. Given that we already have an actor which does most of what we need it feels unnecessary to write a whole new actor from scratch, so I’d like to take the existing SendEmailAfterSubmissionActor as a base for my templated version and add in the functionality to allow the editor to choose from a list of available templates for the email.

Let’s tackle the easy bit first – managing the email templates. We could upload these as files in our solution the same way as we do for views but I can envisage many situations where we’d need to make minor tweaks to the templates which couldn’t really wait for a deployment cycle so I’m going to manage the email templates as content and, as they won’t be accessed directly via a URL, it makes sense to use a block like this.

[ContentType(DisplayName = "Email Template", GUID = "03daf669-5884-47f2-840f-04056afc9536", Description = "Content type for adding email templates")]
public class EmailTemplateBlock : BlockData
{
    [CultureSpecific]
    [Display(
        Name = "Content",
        Description = "The HTML/Text content of the email, use #CONTENT# to represent the editable section of the email",
        GroupName = SystemTabNames.Content,
        Order = 20)]
    [UIHint(UIHint.Textarea)]
    public virtual string Content { get; set; }
}

The slight downside of this approach is that blocks are usually previewed in situ within a page so we need to give our EmailTemplateBlock its own preview controller to allow us to preview the email templates without the site header & footer wrapped around them.

[TemplateDescriptor(
    Inherited = true,
    TemplateTypeCategory = TemplateTypeCategories.MvcController, //Required as controllers for blocks are registered as MvcPartialController by default
    Tags = new[] { RenderingTags.Preview, RenderingTags.Edit },
    AvailableWithoutTag = false)]
[VisitorGroupImpersonation]
[RequireClientResources]
public class EmailTemplatePreviewController : ActionControllerBase, IRenderTemplate<EmailTemplateBlock>
{
    public EmailTemplatePreviewController()
    {
    }

    public ActionResult Index(EmailTemplateBlock currentContent)
    {
        // Display the template with some dummy content
        return Content((currentContent.Content ?? string.Empty).Replace("#CONTENT#", "<p><strong>Lorem ipsum dolor sit amet</strong>,<br/> consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>"));
    }
}

Next, we need to modify the UI for the SendEmailAfterSubmissionActor to allow the user to pick a template along with the rest of the details. This is done in a similar way to code-first content types in that we create a class and decorate its properties to say how we’d like them displayed. To avoid reinventing the wheel, I’m going to inherit from the model used by the default SendEmailAfterSubmissionActor and just add my additional “EmailTemplateId” field which uses a selection factory to show as a dropdown.

public class TemplatableEmailTemplateActorModel : EmailTemplateActorModel
{
    [Display(
        Name = "Template",
        Order = 9999)]
    [SelectOne(SelectionFactoryType = typeof(EmailTemplateSelectionFactory))]
    public virtual int EmailTemplateId { get; set; }
}

public class EmailTemplateSelectionFactory : ISelectionFactory
{
    public IEnumerable<ISelectItem> GetSelections(ExtendedMetadata metadata)
    {
        yield return new SelectItem { Text = "None", Value = 0 };

        var contentModelUsage = ServiceLocator.Current.GetInstance<IContentModelUsage>();
        var contentTypeRepo = ServiceLocator.Current.GetInstance<IContentTypeRepository>();
        var templateType = contentTypeRepo.Load("EmailTemplateBlock");

        var content = contentModelUsage.ListContentOfContentType(templateType) //Get all instances of EmailTemplateBlock
            .GroupBy(x => x.ContentLink.ID).Select(x => x.First()) //Remove duplicates
            .OrderBy(x => x.Name); //Order alphabetically by name

        foreach (var item in content)
        {
            yield return new SelectItem { Text = item.Name, Value = item.ContentLink.ID };
        }
    }
}

Finally, we put it all together by creating our overridden version of the SendEmailAfterSubmissionActor, using our custom model as shown above. In our actor we override the “run” method and use it to modify the model to grab the selected template and wrap it around the email content before putting it back into the email body property of the model and letting the base SendEmailAfterSubmissionActor process the emails as it normally would.

public class CustomSendEmailAfterSubmissionActor : SendEmailAfterSubmissionActor
{
    private Injected<IContentLoader> _contentLoader;
    private string _contentPlaceholder = "#CONTENT#";

    public override Type PropertyType => typeof(PropertyTemplatableEmailActor);

    public override object Run(object input)
    {
        var emailList = Model as IEnumerable<TemplatableEmailTemplateActorModel>;
        foreach (var email in emailList)
        {
            // Load the template in the correct language, allowing for fallbacks
            var languageLoaderOptions = new LoaderOptions();
            languageLoaderOptions.Add(new LanguageLoaderOption { FallbackBehaviour = LanguageBehaviour.FallbackWithMaster, Language = new System.Globalization.CultureInfo((input as EPiServer.Forms.Core.Models.FormIdentity).Language) });
            if (email.EmailTemplateId > 0 && _contentLoader.Service.TryGet(new ContentReference(email.EmailTemplateId), languageLoaderOptions, out EmailTemplateBlock emailTemplateBlock) && email.Body != null)
                {
                email.Body = new XhtmlString(emailTemplateBlock.Content.Replace(_contentPlaceholder, email.Body.ToHtmlString()));
            }
        }
        return base.Run(input);
    }
}

/// <summary>
/// Property definition for the Actor
/// </summary>
[EditorHint("TemplatableEmailActorPropertyHint")]
[PropertyDefinitionTypePlugIn(DisplayName = "TemplatableEmail")]
public class PropertyTemplatableEmailActor : PropertyGenericList<TemplatableEmailTemplateActorModel> { }


/// <summary>
/// Editor descriptor class, for using Dojo widget CollectionEditor to render.
/// Inherit from <see cref="CollectionEditorDescriptor"/>, it will be rendered as a grid UI.
/// </summary>
[EditorDescriptorRegistration(TargetType = typeof(IEnumerable<TemplatableEmailTemplateActorModel>), UIHint = "TemplatableEmailActorPropertyHint")]
public class ConfigurableActorEditorDescriptor : CollectionEditorDescriptor<TemplatableEmailTemplateActorModel>
{
    public ConfigurableActorEditorDescriptor()
    {
        // N.B. This is a special ClientEditingClass just for the EmailTemplateActorModel.
        // Using the expected value of "epi-forms/contentediting/editors/CollectionEditor" will display incorrectly
        ClientEditingClass = "epi-forms/contentediting/editors/EmailTemplateActorEditor";
    }
}

At this point our custom actor will display in addition to the inbuilt one so we’ll want to hide the inbuilt one. We can do this using an EditorDescriptor to override metadata.ShowForEdit as shown below however, before you do this, make sure you've cleared out any existing email responses which have been set up in the inbuilt actor as, once it's hidden, we won't be able to modify any existing emails.

[EditorDescriptorRegistration(
        TargetType = typeof(IEnumerable<EmailTemplateActorModel>),
        UIHint = "EmailTemplateActorEditor",
        EditorDescriptorBehavior = EditorDescriptorBehavior.OverrideDefault)]
public class EmailTemplateActorEditorDescriptor : CollectionEditorDescriptor<EmailTemplateActorModel>
{
    public EmailTemplateActorEditorDescriptor()
    {
        //ClientEditingClass = "epi-forms/contentediting/editors/EmailTemplateActorEditor";
    }
    public override void ModifyMetadata(ExtendedMetadata metadata, IEnumerable<Attribute> attributes)
    {
        metadata.ShowForEdit = false;
    }
}

And there we have it, templated emails sent through Episerver Forms.

As always, I’ve created this as a proof-of-concept rather than battle-hardened, production-ready code so use with caution. The full source can be found on GitHub here:
https://gist.github.com/PaulGruffyddAmaze/6ce76f3c6ffb78f0b568c3a2f0998c06

Customising rendering to reduce nested blocks

Mon, 24 Jun 2019 13:55:13 GMT

There have been a couple of posts about performance over the last few weeks from Stefan Holm Olsen then Khurram Khan, both of which suggest minimising the use of nested blocks. This is good advice not just for performance reasons but for improving the editing experience. As developers working in content management, it’s our role to make life easier for the editors who have to use what we implement on a daily basis so, if we can put in a little more work during development to save a lot of content management effort in the future I’d class that as a win.

One of the suggested methods to avoid nesting is to use list properties but, while these are certainly useful and represent the right option in some circumstances, there are drawbacks to be aware of. From an editor’s perspective, using list properties can cause problems in that the items in list properties aren’t reusable in different lists, they’re not well suited to large amounts of fields or large amounts of text in a field, they can’t have personalisation applied and, perhaps most importantly, they are very difficult to manage if you need to translate them between different language branches. From a developer’s perspective, you also need to consider the fact that these property types aren’t yet officially supported (as you can see from the great big warning at the top of the documentation).

One of the most common (and unnecessary) uses for nesting blocks I’ve seen in Episerver implementations is to create a “wrapper” block to group together “item” blocks. To give an example, you might be adding accordion functionality to a site and need to add markup like this:

<div class="accordion">
    <div class="accordion__items">
        <div class="accordion__item">
            <h2 class="accordion__item-heading">Accordion 1</h2>
            <div class="accordion__item-body">
                <p>Accordion content goes here</p>
            </div>
        </div>
        <div class="accordion__item">
            <h2 class="accordion__item-heading">Accordion 2</h2>
            <div class="accordion__item-body">
                <p>Accordion content goes here</p>
            </div>
        </div>
    </div>
</div>

One of my pet hates is to see this implemented by creating an accordion wrapper block like this.

[ContentType(DisplayName = "Accordion Wrapper", GUID = "C11E7439-DD80-488B-8CEB-23FA6D23508D", Description = "Wrapper for accordion items")]
public class AccordionWrapperBlock : SiteBlockData
{
    [Display(
        Name = "Accordion Items",
        Description = "The items in the accordion",
        GroupName = SystemTabNames.Content,
        Order = 10)]
    [AllowedTypes(typeof(AccordionItemBlock))]
    public virtual ContentArea AccordionItems { get; set; }
}

In this instance, the wrapper block exists only to render a wrapping element around the accordion items, adding to the workload of the editors without really adding value. Instead, we should identify groups of similar items while we’re rendering the content area and add in the wrappers programmatically, allowing the editors to concentrate on their content without having to worry about what it should be wrapped in.

In order to address this scenario, first we need to be able to derive which items require which wrapping markup. To do this I’ve created a class which defines how a group should be rendered (ContentAreaWrapper) and an interface which my content types will implement which simply defines which ContentAreaWrapper we should use for that content.

//Define the markup wrapping a group of blocks
public class ContentAreaWrapper
{
    public string StartHtml;
    public string EndHtml;
}

//Interface to allow content types to define how they should be grouped
interface IWrapable
{
    ContentAreaWrapper Wrapper { get; }
}

To ensure consistency and to allow different content types to sit together in the same group, I’ve created a class which contains static instances of my ContentAreaWrapper definitions.

public class ContentAreaWrappers
{
    //Used if no other wrapper is defined
    public static ContentAreaWrapper DefaultWrapper = new ContentAreaWrapper
    {
        StartHtml = "",
        EndHtml = ""
    };

    //Wrapper for accordion blocks
    public static ContentAreaWrapper AccordionWrapper = new ContentAreaWrapper
    {
        StartHtml = "<div class=\"accordion\"><div class=\"accordion__items\">",
        EndHtml = "</div></div>"
    };
}

Rendering the content areas is, unsurprisingly, handled by an instance of ContentAreaRenderer. Sites based on Alloy will already have a customised ContentAreaRenderer registered but, for sites which don’t, it’s just a matter of creating a class which inherits ContentAreaRenderer, overriding the necessary methods and using dependency injection to swap out the default instance of ContentAreaRenderer for your custom version.

In order to wire-in the wrapping of elements, we need to override the RenderContentAreaItems method which is responsible for iterating through the content area items and sending them to be rendered. In this method we’ll loop through the items in the ContentArea, grouping them by their wrapper then sending them to be rendered in those groups, wrapping them in the appropriate HTML.

public class WrappingContentAreaRenderer : ContentAreaRenderer
{
    //Override RenderContentAreaItems to group items together and render in batches
    protected override void RenderContentAreaItems(HtmlHelper htmlHelper, IEnumerable<ContentAreaItem> contentAreaItems)
    {
        if (contentAreaItems == null || !contentAreaItems.Any())
        {
            return;
        }

        var items = new List<ContentAreaItem>();
        ContentAreaWrapper wrapper = ContentAreaWrappers.DefaultWrapper;
        foreach (var item in contentAreaItems)
        {
            var content = item.GetContent();
            var itemWrapper = (content as IWrapable)?.Wrapper ?? ContentAreaWrappers.DefaultWrapper;
            if (itemWrapper != wrapper)
            {
                RenderItemGroup(htmlHelper, items, wrapper);
                wrapper = itemWrapper;
            }
            items.Add(item);
        }
        RenderItemGroup(htmlHelper, items, wrapper);
    }

    //Render grouped items wrapped in appropriate markup
    private void RenderItemGroup(HtmlHelper htmlHelper, List<ContentAreaItem> contentAreaItems, ContentAreaWrapper wrapper)
    {
        if (contentAreaItems.Any())
        {
            htmlHelper.Raw(wrapper.StartHtml);
            base.RenderContentAreaItems(htmlHelper, contentAreaItems);
            htmlHelper.Raw(wrapper.EndHtml);
            contentAreaItems.Clear();
        }
    }
}

Finally, we can create an AccordionBlock implementing IWrapable to define how we should render a group of accordion blocks without requiring a wrapper.

[ContentType(DisplayName = "Accordion Item", GUID = "18d351e2-0046-4421-b405-e14f4ac56bd8", Description = "")]
public class AccordionItemBlock : SiteBlockData, IWrapable
{
    #region Properties
    [CultureSpecific]
    [Display(
        Name = "Heading",
        Description = "The heading of the accordion item",
        GroupName = SystemTabNames.Content,
        Order = 10)]
    public virtual string Heading { get; set; }

    [CultureSpecific]
    [Display(
        Name = "Content",
        Description = "The content of the accordion item",
        GroupName = SystemTabNames.Content,
        Order = 20)]
    public virtual XhtmlString MainContent { get; set; }
    #endregion

    #region IWrapable
    public ContentAreaWrapper Wrapper => ContentAreaWrappers.AccordionWrapper;
    #endregion
}

Obviously this is a fairly simple example to demonstrate the technique but it's be pretty straightforward to extend it to handle more complex scenarios. It's also worth mentioning that, if you're currently adding in row blocks to wrap your inner blocks in a row container you should take a look at the EPiBootstrapArea project from Valdis Iljuconoks which uses a similar technique coupled with display options to dynamically wrap the appropriate number of items in row markup.

Optimising your site for a global audience

Thu, 02 May 2019 12:41:51 GMT

I’d hope most people reading this understand the importance of web performance but, when considering how best to expand an existing site into new markets or to build a site spanning multiple regions, it’s easy to get caught up in the practicalities of localisation, etc. while neglecting the basics. High on the list of priorities for any site should be ensuring the site performs consistently well in all of the countries and regions it serves (not just where the main stakeholders live 😉) which, as I mentioned in a previous post, is not quite as straightforward as it seems, so I put together this post outlining some of the considerations and options available when building a site on a global scale.

Choose your location wisely

Probably the most commonly implemented of the techniques listed here (often combined with a CDN) is to simply choose a single central location to host your site. You take a look at where your traffic comes from and choose a location that best serves that traffic. This choice often comes down to what feels right though you can take a more scientific approach by estimating the percentage of traffic which would be best served by each of the regions in question, estimating the latency between the regions and calculating the impact of putting your site in each region. As an example, the traffic to a site breaks down to 50% EU west, 20% US East and 30% US West. Latency between the datacentres is as follows:

	EU west	US East	US West
EU west	0	73ms	150ms
US East	73ms	0	70ms
US West	150ms	70ms	0

If we multiply the latency by the percentage of traffic, we can get a relative measure of the best location to choose where the lowest number is the best

	EU west	US East	US West	Total
EU west	0 x 50 = 0	73ms x 20 = 1460	150ms x 30 = 4500	5960
US East	73ms x 50 = 3650	0 x 20 = 0	70ms x 30 = 2100	5750
US West	150ms x 50 = 7500	70ms x 20 = 1400	0 x 30 = 0	8900

Use caching effectively

To make the most of your site you should always consider how best to cache your content. Though the term caching may refer to either server-side caching or client-side caching, in this instance I’m talking about client-side caching. It’s not that server-side caching isn’t important, far from it, it’s that server-side caching is as important for an intranet site serving 50 people sat within a 20 metre radius of the server as it is for a global site serving millions of users around the world. One of the main challenges we face when delivering a site globally is network latency. The closer we can get to the browser we’re serving files to, the better and you don’t get much closer than the user’s machine itself. The more we cache on the user’s machine, the less outbound requests we’re making and the lower the impact of the network latency.

At its most basic, we can set cache headers on static assets, including assets like images uploaded to Episerver. To make the most of this cache we can set the expiry to a date in the future (say, 30 days) which means that, all being well, we shouldn’t need to call back to the server until the cache expires but there’s a catch. What if one of those cached files changes an hour after it’s been cached? In theory you’re stuck with it until the cache expires but in practice there’s a few things we can do. If we can encode a timestamp into the URL then, when the asset changes, the URL changes too meaning that the new URL won’t be cached and the file will be requested as though it were new. While this may sound complex to implement, the hard work’s already been done here:

https://github.com/bjuris/EPiServer.CdnSupport

Just add that module and a little bit of config and you need never worry about cache invalidation again.

Beyond simply setting caching headers on static assets, you may also want to consider how best to cache your page content. This can be a problem if your pages are entirely dynamic or heavily personalised however if your content is primarily the same for all users but with one or two personalised areas such as a shopping cart count/value or logged in user’s name, it may be better to load a cacheable generic version of the page without those parts then load in the dynamic parts using JavaScript.

Finally, if you need finer grained control over how you cache or load your content and assets, you may want to consider using service workers to manage a local cache or pre-fetching content and assets using a <link rel=”prefetch” /> tag.

Implement a CDN

If I could recommend just one of the techniques on this page it would be this one. Seriously, just do it. Even if you’re just serving a single region a content delivery network can dramatically improve a site’s performance (if you use it correctly). A CDN builds on the browser caching techniques described above by caching your content at edge locations closer to your end users. This means that you get a double-whammy performance boost in that there’s less latency between the user and the content and, just as importantly, your server isn’t having to handle requests for content cached in the CDN, freeing up resources to process requests which can’t be cached on the CDN.

There was a time not so long ago when using a CDN was pretty pricey but nowadays you can implement a CDN for almost nothing on the likes of Azure, AWS CloudFront and CloudFlare, or for literally nothing on CloudFlare’s free plan (though with some limitations). As if that wasn’t enough of an incentive, these CDNs often offer additional services either bundled or for a small additional cost. As an example, the paid CloudFlare plans (as included in Episerver DXC service) go beyond the basic caching proxy functionality of a traditional CDN and add DDoS protection, Web Application Firewall and image optimisation (e.g. automatically serving images in webp for browsers which support it), making your site even faster and more resilient.

Separate sites

While this option is simple and effective, there are many drawbacks. This technique involves setting up entirely separate Episerver instances, each running a site to serve a different location. You may choose to run them all on a shared codebase to make overall management easier but you’ll get none of the benefits of running a single instance covering multiple locales. As an example, there would be no option for a language to fall back to another if they exist in different Episerver instances and similarly adding hreflang tags to link between languages becomes a manual process which is near impossible to keep on top of. This will also have a cost impact in terms of Episerver licensing and infrastructure costs.

What you do get from this approach is a simple method for setting up sites with infrastructure close to your end users so, if your country sites are entirely managed by local teams and don’t necessarily have to share a common structure or interlink, this may work for you but I suspect that’s a pretty rare scenario these days.

Mirroring

A slight variation on the separate sites approach would be a technique I’m going to refer to as mirroring. Why? Well, back in the days when Episerver had to be installed through “Deployment Centre”, there was a feature called mirroring which allowed you to create content in one environment and transfer it to a different environment using a synchronisation service. Fast forward to 2018 and Episerver’s Mark Hall created a content synchronisation mechanism which gives you the benefits of mirroring while addressing some of the issues of the official (though deprecated) Episerver mirroring feature.

https://world.episerver.com/blogs/Mark-Hall/Dates/2018/6/simple-content-synchronization/

Though the solution here has been designed to allow a configuration without a CMS UI in the production environment, you could just as easily use it to create content in a single location and push it out to Episerver instances closer to your site users. Though this solves the shared/linked content issue from the separate sites approach, be aware that there will still be the same licensing implications.

Replication

If you’re running in Azure and you’re willing to accept the limitation of having only a single writeable region and having all others read-only then you could consider taking advantage of Azure SQL’s geo replication capabilities. This technique allows read-only replicas of your database to be maintained in different azure regions which, when coupled with app service instances in those regions, allows for duplicate but synchronised infrastructure which can be scaled out across regions. Unfortunately there are a few complications with this approach but, at the risk of being accused of blatant self-promotion, I wrote a blog post explaining how to use the existing components of an Episerver site in Azure to work around these complications.

https://world.episerver.com/blogs/paul-gruffydd/dates/2018/10/going-global--geographically-scaling-an-episerver-site-over-multiple-azure-regions/

Decoupling

So far we’ve looked at techniques which can be applied to an existing site with minimal changes but what if you’ve got a blank canvas? One option would be to look at decoupling the CMS from the website so that the CMS sits in a single location and is used simply as a data source with the rendering and serving of the content happening independently in locations closer to your end users.

With the advent of the Episerver Content API, this has opened up various options for us in terms of how we use content produced in the CMS with probably the most popular being to use single page app techniques to read data from the CMS and render it using libraries such as React, Vue or [insert name of JavaScript library-du-jour]. There are plenty of articles out there which explain the benefits of this approach so, in the interests of balance I feel I should point out that single page apps aren’t a silver bullet. Without effective planning and caching between the app and the Episerver instance, this approach has many of the same pitfalls as simply rendering the site using razor on the server in that the data still needs to be requested from the Episerver instance and that data will be subject to the same network latency we were trying to avoid. Done well though, you can get some great results.

Taking a slightly more extreme approach, Allan Thraen’s post explains how you can publish static HTML out of an Episerver instance and, as I think we can all agree, nothing scales quite as well as static HTML. As with the replication option above, this technique results in a read-only site and, if you’ve got hundreds of thousands of pages of constantly changing content this may not be the right option for you however, for smaller, more static sites, this would be easy to scale out and cost next to nothing to host.

https://www.codeart.dk/blog/2018/9/episerver-static-web-site-generator/

What next?

If you've made it this far - well done. Hopefully you've got an idea of some of the options currently available to you either individually or in combination but you’d be forgiven for thinking that this all seems like a lot of work and that there should be just a setting hidden away in Episerver admin that sorts this for you. Well, though it’s not going to happen overnight, there are some interesting developments in this area. You may have noticed that most, if not all of Episerver’s newer products and features have been built as independent hosted services and that approach is likely to extend beyond new features into the core of the CMS and Commerce.

At last year’s Ascend events, there was talk of a plan to create an externally hosted order service for Episerver commerce to handle the ordering process separately from the installed Episerver site so you can still take orders even when your site is in read-only mode. Fast forward to February this year and that order service had become a reality which was demonstrated at the Partner close-up event in Stockholm. Alongside the order service there were rather more subtle mentions of a hosted content service and, though there are no details of this service beyond describing a suite of independent microservices based on .Net Core, it wouldn’t be unreasonable to assume that it would be built in such a way as to allow for geographical replication. By taking a distributed approach, it reduces the risk from a failure of a single region and paves the way for the next generation of distributed web applications making it a win-win-win for Episerver, partners and customers.

Future-proof your URLs with emojis

Mon, 01 Apr 2019 10:45:48 GMT

They say a picture paints a thousand words and, in our fast-paced world, who’s really got time to read a thousand words? Indeed, it would seem that even the 160 characters of a text message is a struggle for some these days leading to the creation of a whole new language - Emoji. But why should text messages have all the fun while web sites are stuck in the past?

Imagine a world where a website’s cookie policy could have the URL /🍪 rather than /cookies. This not only shrinks the URL to something memorable but also removes any language barriers as everyone can understand a picture regardless of the language they speak. In fact, even when languages should be the same there can be subtle differences which can cause problems. Imagine you ran an english language recipe site targeting the UK and US. You might have a URL like the following:
/recipes/tasty-aubergine-with-a-splash-of wine/

But in the US they don't use the word aubergine, they call it an eggplant so this would need to be rewritten to:
/recipes/tasty-eggplant-with-a-splash-of-wine/

If we were to write this in emoji we could have the following:
/👩‍🍳/😋🍆💦🍾/
Which, as you can see, is fun, concise, easy to remember and, most importantly, couldn’t possibly be misinterpreted.

Time to take a look at how we could do this in Episerver.

If we try to add an emoji without making any modifications, we get this.

This is due to the validation rules put in place by Episerver but, if the last couple of years have shown anything, it’s that us plucky Brits won’t just sit back and accept arbitrary rules put in place by unelected Europeans 😉. Time to take back control…

First we need to modify the rules for validating allowed characters in URL segments which we can do by overriding the UrlSegmentOptions in an IConfigurableModule.

[ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
public class UrlInitialisation : IConfigurableModule
{
    public void ConfigureContainer(ServiceConfigurationContext context)
    {
        //Enable emojis in URLs
        context.Services.RemoveAll<UrlSegmentOptions>();
        context.Services.AddSingleton<UrlSegmentOptions>(s => new UrlSegmentOptions
        {
            Encode = true,
            ValidCharacters = @"\p{L}0-9\-_~\.\$\u00a9\u00ae\u2000-\u3300\ud83c\ud000-\udfff\ud83d\ud000-\udfff\ud83e\ud000-\udfff"
        });
    }


    public void Initialize(InitializationEngine context) {
        //TODO

    }

    public void Uninitialize(InitializationEngine context) {
        //TODO
    }
}

At this stage we can now add emojis to our URLs but I want to take it a step further and get Episerver to generate emoji URL segments. To do this, I’ve downloaded a set of emojis from here which I’ve modified to remove unnecessary info, make the keywords more relevant and to tidy up some of the keywords we’ll use for our replacements. The new format for the emojis is as follows:

[
  {
    "Character": "🧐",
    "Keywords": "monocle|stuffy|rees-mogg"
  },
  {
    "Character": "💩",
    "Keywords": "dung|turd|poo|poop|brexit"
  }

]

Next, I load that into a collection of .net objects using json.net and build a dictionary mapping words to their emoji replacement. Then finally I’ve hooked in to the IUrlSegmentCreator’s Created event where I loop through the dictionary replacing any words I can.

Putting it all together we get this:

[ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
public class UrlInitialisation : IConfigurableModule
{
    private Injected<IUrlSegmentCreator> _urlSegmentCreator;
    private Dictionary<string, string> _emojis = new Dictionary<string, string>();

    public void ConfigureContainer(ServiceConfigurationContext context)
    {
        //Enable emojis in URLs
        context.Services.RemoveAll<UrlSegmentOptions>();
        context.Services.AddSingleton<UrlSegmentOptions>(s => new UrlSegmentOptions
        {
            Encode = true,
            ValidCharacters = @"\p{L}0-9\-_~\.\$\u00a9\u00ae\u2000-\u3300\ud83c\ud000-\udfff\ud83d\ud000-\udfff\ud83e\ud000-\udfff"
        });
    }


    public void Initialize(InitializationEngine context) {
        SetupEmojis();
        _urlSegmentCreator.Service.Created += CreateUrlSegment;

    }

    public void Uninitialize(InitializationEngine context) {
        _urlSegmentCreator.Service.Created -= CreateUrlSegment;
    }

    private void CreateUrlSegment(object sender, UrlSegmentEventArgs e)
    {
        foreach (var keyword in _emojis.Keys.OrderByDescending(x => x.Length))
        {
            //TODO: Remove stop words
            e.RoutingSegment.RouteSegment = e.RoutingSegment.RouteSegment.Replace(keyword, _emojis[keyword]);
        }
        //remove dashes
        e.RoutingSegment.RouteSegment = e.RoutingSegment.RouteSegment.Replace("-", "");
    }

    private void SetupEmojis()
    {
        var emojis = JsonConvert.DeserializeObject<List<EmojiInfo>>(File.ReadAllText(HttpContext.Current.Server.MapPath(@"/App_Data/emojis.json")));
        foreach (var emoji in emojis)
        {
            foreach (var keyword in emoji.KeywordList)
            {
                if (!_emojis.ContainsKey(keyword))
                {
                    _emojis.Add(keyword, emoji.Character);
                }
                var plural = keyword + "s";
                if (!_emojis.ContainsKey(plural))
                {
                    _emojis.Add(plural, emoji.Character);
                }
            }
        }
    }
}

public class EmojiInfo
{
    private IEnumerable<string> _keywordList = null;
    public string Character { get; set; }
    public string Keywords { get; set; }

    public IEnumerable<string> KeywordList
    {
        get
        {
            return _keywordList ?? Keywords.Split('|').Select(x => x.Replace(" ","-"));
        }
    }
}

So, why not join the 😎 kids and add emojis to your URLs today or are you 🕑😰?

Feature switching based on language branch

Wed, 06 Mar 2019 13:51:13 GMT

Look after a multi-lingual site for long enough and there will come a point where you need to roll out some functionality one language or region at a time. At the recent developer meetup in London I coughed my way through presenting a technique for doing just that.

The idea came from a client requirement where they have a multi-lingual site run by local teams in those markets. Their requirement involved a fairly extensive third party integration but, because of the nature of that integration it needed to be rolled out one market at a time. From the CMS side, we needed to add a number of fields to a number of our content types and ultimately provide a checkbox which determines whether the new integration is displayed on the site.

In theory that's pretty straightforward but it’s not the best editor experience to have a bunch of fields which won’t actually do anything if you’re not in one of the chosen languages the functionality has been rolled out to. My first thought was to create a validator which could be applied to the feature-specific fields and check whether the feature has been enabled for that language. If the feature hasn’t been enabled and an editor starts typing into one of the fields, they will be shown an error message letting them know that the functionality hasn’t been rolled out to their region. This approach is better than nothing but still isn’t the best experience for the editors as we’re still, in effect, offering them a feature they can’t have. For the best editor experience, we should only show the feature-specific fields if that feature is enabled but, as all of our languages use the same content types, how can we set about doing that?

My initial assumption was that, if we need to make a change to the way the Episerver editor displays a field, we’d need to do something with dojo and, quite frankly, every time it looks like I’ll need to write some dojo my heart sinks. In this instance however I found an alternative.

When looking into the validator approach I noticed that, on a validation attribute (or, in fact, any attribute), you can implement the IMetadataAware interface which requires an additional method called OnMetadataCreated. What this method does is to allow us to modify the settings which tell the Episerver editor UI how to display a given field before the field is rendered and, perhaps most interestingly in this case, gives us the option to mark the field as read-only or to hide it from view altogether.

So, now onto some code. In order to keep things simple, the code below is a kind of simplified version of what was produced, built on the alloy site which you should all be familiar with. In the code below, our integration is a twitter widget which requires us to set a heading and an account to pull the feed from.

First up we need a mechanism to determine whether the integration is available in a given language and there are many ways to do this. If you’re after a more comprehensive feature switching library, check out this project from Valdis but, for the sake of simplicity, in the example below I’ve just used a Boolean field in a tab called “features” on the start page of the site. The reason for the special tab is twofold. First, it allows us some scope for extending this functionality to other features in future – we just need to add an additional Boolean field for each feature and they’re all grouped together. The other reason is so we can apply permissions to the tab to stop local editors from enabling features they shouldn’t have access to.

[Display(Name = "Activate Twitter Functionality", GroupName = Global.GroupNames.Features)]
[CultureSpecific]
public virtual bool TwitterFeedActive { get; set; }

Next we create an attribute implementing IMetaDataAware which accepts the name of a feature, checks to see whether the feature is enabled in the current language and, if it’s not, modifies the field’s configuration to hide it from the editor.

public class FeatureSwitch : Attribute, IMetadataAware
{
    private string _featureName;
    private IContentLoader _contentLoader;

    public bool AllowEditing
    {
        get
        {
            if (!_contentLoader.TryGet(ContentReference.StartPage, out StartPage startPage))
            {
                return false;
            }
            //Check whether our feature switch property exists and has been checked on the start page
            return startPage.Property.Any(x => x.Name.Equals($"{_featureName}Active"))
                && ((startPage.Property[$"{_featureName}Active"]?.Value as bool?) ?? false);
        }
    }

    public FeatureSwitch(string featureName)
    {
        _featureName = featureName;
        _contentLoader = ServiceLocator.Current.GetInstance<IContentLoader>();
    }

    public void OnMetadataCreated(ModelMetadata metadata)
    {
        metadata.ShowForEdit = AllowEditing;
    }
}

Finally, we add the validation attribute to all relevant fields, specifying the feature they relate to.

[Display(
    Name = "Twitter Feed Heading",
    GroupName = SystemTabNames.Content,
    Order = 5)]
[CultureSpecific]
[FeatureSwitch("TwitterFeed")]
public virtual string TwitterFeedHeading { get; set; }

[Display(
    Name = "Twitter Account",
    GroupName = SystemTabNames.Content,
    Order = 6)]
[CultureSpecific]
[FeatureSwitch("TwitterFeed")]
public virtual string TwitterAccount { get; set; }

And there we have it, an alloy site with the fields to set up a twitter feed on the EN site but not on the SV site.

Taking this a step further, it may not just be individual properties we want to limit to a specific language, it may be content types as well. Continuing the twitter feed example from above, you may want to add a specific block type to represent the twitter feed rather than just properties on a specific page type so how can we extend this approach to do that as well?

If we have individual teams managing content in each language then we could use permissions on the content types to limit who can create content of that type however this doesn’t allow for scenarios where a given individual works in multiple languages. It’s also a second place for us to make changes which feels unnecessary given that we’ve already got a feature switch defined on our start page.

The better option here is to tie in to the ContentTypeAvailabilityService which is responsible for providing the listings of content types we can use when we click to create a new content item. By taking this approach we can make use of our existing FeatureSwitch attribute.

We could create our own implementation of the abstract ContentTypeAvailabilityService class but that would involve writing our own version of the existing logic used to retrieve and filter the content types which seems like a lot of work just to filter the list of content types. We could also consider inheriting from the DefaultContentTypeAvailabilityService and just overriding the bits we need but that is found in an internal namespace and so is prone to breaking changes without warning so, instead I’m going to intercept whichever implementation of the ContentTypeAvailabilityService is being used and add the filtering on top.

First, we need to create a class inheriting from ContentTypeAvailabilityService with a constructor accepting the currently registered instance. Next, in the ListAvailable methods, we take the list generated by the registered ContentTypeAvailabilityService instance and loop through the available content types, checking for the FeatureSwitch attribute. If we find a content type with that attribute, we check the AllowEditing property on the attribute and, if it’s false, we remove the type from the list.

public class CustomContentTypeAvailabilityService : ContentTypeAvailabilityService
{
    private readonly ContentTypeAvailabilityService _defaultContentTypeAvailabilityService;
    public CustomContentTypeAvailabilityService(ContentTypeAvailabilityService defaultContentTypeAvailabilityService)
    {
        _defaultContentTypeAvailabilityService = defaultContentTypeAvailabilityService;
    }
    public override AvailableSetting GetSetting(string contentTypeName)
    {
        return _defaultContentTypeAvailabilityService.GetSetting(contentTypeName);
    }

    public override bool IsAllowed(string parentContentTypeName, string childContentTypeName)
    {
        return _defaultContentTypeAvailabilityService.IsAllowed(parentContentTypeName, childContentTypeName);
    }

    public override IList<ContentType> ListAvailable(string contentTypeName, IPrincipal user)
    {
        return FilterAvailableList(_defaultContentTypeAvailabilityService.ListAvailable(contentTypeName, user));
    }

    public override IList<ContentType> ListAvailable(IContent content, bool contentFolder, IPrincipal user)
    {
        return FilterAvailableList(_defaultContentTypeAvailabilityService.ListAvailable(content, contentFolder, user));
    }

    private IList<ContentType> FilterAvailableList(IList<ContentType> list)
    {
        for (var i = list.Count-1; i >= 0; i--)
        {
            var contentType = list[i];
            var attributes = contentType.ModelType.GetCustomAttributes(true) ?? new object[0];
            foreach (var attribute in attributes)
            {
                var featureAttribute = attribute as FeatureSwitch;
                if (featureAttribute != null && !featureAttribute.AllowEditing)
                {
                    list.RemoveAt(i);
                }
            }
        }
        return list;
    }
}

Next, we need to register our class to intercept the existing ContentTypeAvailabilityService implementation. We can do this in an initialisation module as follows.

[ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
public class ContentTypeAvailabilityInitialization : IConfigurableModule
{
    public void ConfigureContainer(ServiceConfigurationContext context)
    {
        context.Services.Intercept<ContentTypeAvailabilityService>(
            (locator, defaultContentTypeAvailabilityService) => new CustomContentTypeAvailabilityService(defaultContentTypeAvailabilityService));
    }
    public void Initialize(InitializationEngine context) { }
    public void Uninitialize(InitializationEngine context) { }
}

Then finally we can mark some content types in the same way as we marked our properties and verify that they only appear on languages where that feature is enabled.

/// <summary>
/// Used to add a twitter feed
/// </summary>
[SiteContentType(GUID = "D8C3E587-AA5D-4456-8BC7-47CF971CC835")]
[SiteImageUrl]
[FeatureSwitch("TwitterFeed")]
public class TwitterFeedBlock : SiteBlockData
{
    [Display(
        Name = "Twitter Feed Heading",
        GroupName = SystemTabNames.Content,
        Order = 5)]
    [CultureSpecific]
    public virtual string TwitterFeedHeading { get; set; }

    [Display(
        Name = "Twitter Account",
        GroupName = SystemTabNames.Content,
        Order = 6)]
    [CultureSpecific]
    public virtual string TwitterAccount { get; set; }
}

As a final note, this technique isn’t limited to just restricting by language, you could easily use the same approach to handle multi-site scenarios, differing editor permissions or even more outlandish requirements like “we should only be able to add blog articles on a Friday afternoon”. Whatever the requirements, I hope you find it useful.

Limiting total items in a content area while supporting personalisation

Tue, 30 Oct 2018 14:06:20 GMT

As anyone who’s spoken to me at any length about Episerver can confirm, I’m something of a fan of Episerver’s inbuilt visitor group personalisation capabilities. It’s easy to use and, done right, can add a load of value for next to no effort but all too often I see it forgotten about during builds. The most common occurrence of this that I see is the use of a ContentArea’s Items collection rather than FilteredItems but I was reminded of another by Aniket’s recent(ish) blog post.

It’s a technique I’ve seen both in various blog posts and out in the wild being used on sites but, while it does exactly as it claims (sets a maximum number of items you’re allowed to add to a content area), it’s probably not what you need. Why? Take a look…

In this example we’ve got a content area for a hero carousel which can display no more than 3 blocks. To enforce this, I’ve added the validation attribute described in the posts linked above to limit the maximum number of items in the content area to 3. I’ve then set up some blocks so that only 3 will display. All good?

Well, not really. Even though only 3 items will display, the validation attribute will throw a validation error because it doesn’t consider personalisation. If we remove items until it stops throwing an error, in this instance we’d end up with only a single item being displayed on the site.

To fix the issue we could use FilteredItems as I mentioned above and that would kind of work but then consider this scenario…

As the editor will always be logged in and the validation will run in the context of the editor, the first item would be filtered out and so, even though we could quite feasibly return 4 items from the FilteredItems collection for a user viewing the site, the validation will let that through.

A better alternative would be to look at each of the content area items in the Items collection and check whether they’re part of a personalisation group. We can then total up the number of groups plus the number of items which aren’t in a group and that will give us the maximum number of items in the content area after they have been filtered.

Putting it all together, we get this:

/// <summary>
/// Sets the maximum item count in a content area once personalisation is applied
/// </summary>
[AttributeUsage(AttributeTargets.Property, AllowMultiple = false)]
public class MaxItemsAttribute : ValidationAttribute
{
    private int _maxAllowed;

    public MaxItemsAttribute(int MaxItemsAllowed)
    {
        _maxAllowed = MaxItemsAllowed;
    }

    protected override ValidationResult IsValid(object value, ValidationContext validationContext)
    {
        var contentArea = value as ContentArea;

        // Get all items or none if null
        var allItems = contentArea?.Items ?? Enumerable.Empty<ContentAreaItem>();

        // Count the unique personalisation group names, replacing empty ones (items which aren't personalised) with a unique name
        var i = 0;
        var maxNumberOfItemsShown = allItems.Select(x => string.IsNullOrEmpty(x.ContentGroup) ? (i++).ToString() : x.ContentGroup).Distinct().Count();

        return (maxNumberOfItemsShown > _maxAllowed) ? new ValidationResult($"The property \"{validationContext.DisplayName}\" is limited to {_maxAllowed} items") : null;
    }

}

Which we can use like this:

[Display(Name = "Hero Carousel")]
[AllowedTypes(typeof(HeroItem))]
[MaxItems(3)]
public virtual ContentArea HeroCarousel { get; set; }

Geographically scaling an Episerver site over multiple Azure regions

Tue, 02 Oct 2018 12:20:31 GMT

There are many reasons why you might want to run your site out of multiple regions – performance, SEO, disaster recovery, legislative or even just preference. In our current era of dev-ops and cloud technology, it’s become almost trivial to simultaneously deploy a codebase to servers around the world so you’d be forgiven for thinking that deploying an Episerver site to several global locations would be just as simple but there’s a catch. Behind every modern Episerver site is a SQL server database and, for all its plus points (and there are many), geographical distribution isn’t really SQL server’s forté. This means that you’re basically stuck with your database being hosted in a single location and, because your web servers need the fastest access they can to the DB, that also means that your web servers are limited to that same location.

Azure SQL to the rescue?

Unwilling to accept that limitation, I thought I’d take a look at Azure SQL to see if I could find a workaround and, though it shares a common heritage with SQL server, Azure SQL does have a couple of features which look like they may be of use.

First up, “SQL Data Sync”. This feature is mainly for managing hybrid infrastructure so it’s really targeted at DBAs used to setting up SQL server replication but what it gives us is multiple writeable clones of our database which are magically sync’d by a separate task. But that’s where the good news stops. If you take a look at the FAQs, alarm bells should start to ring. First of all, you need to tell it what to sync down to a column level in your database tables which, aside from taking forever to set up, is likely to cause problems in the future should the schema change as a result of upgrades, etc. There’s also the issue of timings. The synchronisation task happens on a schedule with a frequency between 5 minutes and 30 days which means that, even if you run the sync as frequently as possible, it may be 5 minutes before a change is synchronised and even longer before it would be visible on the site due to the caching within the Episerver DataFactory. In summary, you could probably make this route work but you’d most likely spend the remainder of your life wishing you hadn’t.

Our other option is to use geo-replication. While this feature was primarily built for disaster recovery scenarios, it is much closer to what we need in that the synchronisation is a continuous process performed as the data changes rather than on a schedule. It’s also worth mentioning at this point that geo-replication in Azure SQL is really easy to set up. You basically click “Geo-Replication” on your database in Azure Portal, click on the region you want to replicate your database to and choose the characteristics of the database then just sit back and wait while Azure handles the rest.

But there’s a catch. Because this mechanism was built for DR/failover scenarios, we’re left with a fully functioning primary database but our replicas are read-only. That said, as long as you don’t need to write to the Episerver Database from your remote sites, you can simply switch your site into read-only mode.

So far, so good but what happens when you publish content in the CMS? Well, as you’ve only got 1 writable database, all of your editing would need to happen in your primary region. When you hit publish, the status of the piece of content is updated in the database and the publish event is fired. The remote site picks up that event and clears the relevant record from its cache to be rebuilt on the next request which would be fine if the database synchronisation completes before the event is fired but it doesn’t. The synchronisation happens asynchronously so there’s a slight delay between the primary returning to say it’s updated and the replica being updated. This delay is typically less than a few seconds but is just enough to cause us problems. If someone requests the updated page on the remote site between the primary updating and the replica receiving the update, the cache on the remote site will be rebuilt with the stale data and our change won’t be visible until that cache is cleared again so what we ideally need to do is to hold off on clearing the remote cache until the data has been replicated.

Delayed events

Just to recap on the remote events process as it stands for sites on Azure – an event is raised by one instance of the site which adds details of the event to a message sent to a topic within Azure Service Bus. All instances of the site are subscribed to the topic and so each one receives and acts on the message. If we want to add in a delay for some subscribers, our easiest route is to create a second topic which the remote instances can subscribe to and pass messages from one to the other after the delay has passed.

Given that it there’s no way to know for sure exactly when the relevant data will have been synchronised, we’re going to have to make an assumption as to how long this sync will take and delay the remote events to the remote server by that amount. There are a few ways we could go about doing this (custom event provider, Azure functions, Logic app, etc.) but I like to keep things simple and reuse as much existing functionality as we can.

As luck would have it, Azure Service Bus has some features which will help us with this, namely auto-forwarding, filter rules and message scheduling. By using these, we shouldn’t need any additional code or services to monitor, wait and forward messages as the service bus should be able to handle this all for us.
To get this up and running, I’ve created an initialisation module which sets up the topics (if they don’t exist) and adds a subscriber which will pick up messages and schedule the delivery of each message into the new topic with a slight delay (10 seconds).

    [ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
    public class ServiceBusGeoInitialisation : IInitializableModule
    {
        public void Initialize(InitializationEngine context)
        {
            var connStr = ConfigurationManager.ConnectionStrings["EPiServerAzureEvents"].ConnectionString;
            var topic = ConfigurationManager.AppSettings["LocalTopicName"] ?? "mysiteevents";
            var geoTopic = ConfigurationManager.AppSettings["DelayedTopicName"] ?? "mysiteeventsdelayed";
            var nsMgr = Microsoft.ServiceBus.NamespaceManager.CreateFromConnectionString(connStr);

            //Create our local topic if it doesn't already exist
            if (!nsMgr.TopicExists(topic))
            {
                nsMgr.CreateTopic(topic);
            }

            //Create our new topic if it doesn't already exist
            if (!nsMgr.TopicExists(geoTopic))
            {
                nsMgr.CreateTopic(geoTopic);
            }

            //Create a new subscription to forward messages from our local topic to our new topic with a delay
            if (!nsMgr.SubscriptionExists(topic, "GeoForwarder"))
            {
                var subscription = new SubscriptionDescription(topic, "GeoForwarder")
                {
                    ForwardTo = geoTopic
                };
                nsMgr.CreateSubscription(subscription);

                var subscriptionClient = SubscriptionClient.CreateFromConnectionString(connStr, topic, "GeoForwarder");
                //Clear out default rule
                subscriptionClient.RemoveRule(RuleDescription.DefaultRuleName);

                //Set up a new rule to match all messages and schedule for 10 seconds time
                var rule = new RuleDescription
                {
                    Name = "GeoDelayRule",
                    Filter = new SqlFilter("1 = 1"),
                    Action = new SqlRuleAction("SET sys.TimeToLive = '00:00:10'; SET sys.ScheduledEnqueueTimeUtc = sys.ExpiresAtUtc; SET sys.TimeToLive = '00:30:00'")
                };
                subscriptionClient.AddRule(rule);
            }
        }

        public void Uninitialize(InitializationEngine context)
        {
        }
    }

For those wondering what’s going on in the filter rule, it would appear that Azure service bus SQL syntax doesn’t support any kind of date manipulation functions so I’ve had to find a workaround. In this instance I’m setting the TimeToLive value on the message to 10 seconds which automatically sets the ExpiresAtUtc value. I can then take the value of ExpiresAtUtc and use it to set the ScheduledEnqueueTimeUtc field before resetting the ExpiresAtUtc value back to 30 minutes. Crazy to have to do it that way but it works.

So there you have it. One initialisation module and a bit of config and we’ve got a working Episerver site running happily in multiple Azure regions. There are, of course, other options available which I’ll aim to cover off in a subsequent post but hopefully you’ve found this useful.

As always, the code posted here is simply a proof-of-concept rather than tested, production ready code so use with caution.

Listing popular content with Profile Store

Fri, 08 Jun 2018 10:43:29 GMT

Every so often, a certain requirement comes up. Sometimes it’s called “Most Popular”, sometimes it’s called “Trending”, I’ve even seen it called “Hot topics” but, regardless of the name, the requirement is the same – Show a listing of the top x most popular pieces of content. On paper, this may sound straightforward but it hides bit of a challenge in that, in order to show the most popular pages, you need to know what content is popular which generally requires keeping track of page views within a given timeframe.

One approach to this would be to take advantage of auto boosting within Find to apply a hit boost to a search. This would be pretty straightforward (and pretty efficient) however the hits tracked to apply this boost only come from tracked clicks from the results of a find search. This means that hits from navigation, social sharing, google searches, etc. won't be counted so you may miss out on a fair proportion of the hits for a given page.

An alternative approach would be to pull data from an analytics package such as Google Analytics or, as I've chosen in this instance, Episerver Profile Store. The process would be similar whether we were to use GA or Profile Store but this is a good opportunity to take a look at how we can put the Profile Store to good use beyond simply tracking data and looking at it in Insight.

Tracking

The first thing we need to do is to track our page views and, within profile store, there are many ways we could do this. I could go into detail here but I think that’s been covered fairly comprehensively by David Knipe and Nicola Ayan though, having said that, I’m going to use a slightly different method and use the [PageViewTracking] attribute from the “EPiServer.Tracking.PageView” library available through NuGet. Why? Well, a few reasons. Partly because it’s nice and easy (you just add [PageViewTracking] to your controller action and it does the rest), partly because I couldn’t get the [Tracking()] attribute from Episerver.Tracking.Cms to work, but mostly because we need to track things in a consistent way.

At present, it’s only really Episerver Advance personalisation which requires you to track specific data in the Profile Store and it requires that data in a specific format. As more and more features come to rely on the data in the Profile Store, they too will need this data in a consistent format so, to me at least, it seems reasonable to expect that those features would require data in the same format as Advance to avoid having to raise multiple page view tracking events in slightly different formats on each page request.

Aggregating the data

Now we’ve got data flowing in to our profile store instance, we can look at how we can use that data to power our listing. Profile Store comes with a rest API for querying data both from individual profiles and from the events which were tracked. In this instance we’re going to use the latter to pull out all recent “epiPageView” events (as tracked by “EPiServer.Tracking.PageView”). First of all though, let’s take a look at the structure of the data we’re requesting.

The basic structure of an event tracked by the attribute mentioned above is as follows, where the top level of this object is common to all events tracked within Profile Store but the contents of the Payload can be any arbitrary data we want to add:

{
    "TrackId": null,
    "DeviceId": "bf4b2611-63f5-4364-899a-7017f6d044b5",
    "EventType": "epiPageView",
    "EventTime": "2018-05-31T15:50:38.4951303Z",
    "Value": "Viewed Start",
    "Scope": "463470c3-3eca-41d3-8b12-3f7f92f62d34",
    "CountryCode": "Localhost",
    "PageUri": "http://localhost:59422/",
    "PageTitle": null,
    "RemoteAddress": "127.0.0.1",
    "Payload": {
        "epi": {
            "contentGuid": "bd437cef-41bd-4ebc-8805-0c20fcf4edcf",
            "language": "en",
            "siteId": "463470c3-3eca-41d3-8b12-3f7f92f62d34",
            "ancestors": [
                "43f936c9-9b23-4ea3-97b2-61c538ad07c9"
            ],
            "recommendationClick": null
        }
    },
    "User": {
        "Name": null,
        "Email": ""
    }
}

When we query for this data we get back an array of these objects plus some totals. I’m going to use RestSharp for the Rest requests and Newtonsoft.Json to deserialise my data into a slightly cut-down object representation of the data (shown below):

{
    "Total": 123,
    "Count": 123,
    "Items": [
        {
            "Payload":{
                "epi": {
                    "contentGuid": "00000000-0000-0000-0000-000000000000",
                    "language": "en"
                }
            }
        }
    ]
}

In order to get back the list of popular pages, we need to query Profile Store for “epiPageView” hits within a given timeframe and total the results. The basic URL for this query looks something like this:
/api/v1.0/trackevents/?$filter=EventType eq epiPageView and EventTime gt 2018-05-01T00:00:00Z

In an ideal world, we could do the aggregation as part of our query but unfortunately this isn’t supported right now so we’re going to have to do it ourselves.

Obviously sifting through thousands of hits is not something we want to do on the fly so we'll use a scheduled job. This scheduled job pulls back the epiPageView events in pages of 1000, tallies up the totals and saves the result to the Dynamic Data Store in a suitable format for future consumption, adding in the PageTypeId to allow us to query by PageType later.

    [ScheduledPlugIn(DisplayName = "Index Recent Page Hits", DefaultEnabled = false, IntervalLength = 1, IntervalType = EPiServer.DataAbstraction.ScheduledIntervalType.Hours)]
    public class IndexRecentHits : ScheduledJobBase
    {
        //Settings
        private readonly string _apiRootUrl = ConfigurationManager.AppSettings["episerver:profiles.ProfileApiBaseUrl"];
        private readonly string _appKey = ConfigurationManager.AppSettings["episerver:profiles.ProfileApiSubscriptionKey"];
        private readonly string _eventUrl = "/api/v1.0/trackevents/";
        private readonly string _timeWindow = ConfigurationManager.AppSettings["RecentHours"] ?? "24";
        private readonly int _resultsPerPage = 1000;

        private Dictionary<string, int> _recentHits = new Dictionary<string, int>();
        private bool _stopSignaled;

        private static DynamicDataStoreFactory _dataStoreFactory;
        private static IContentLoader _contentLoader;

        public IndexRecentHits(DynamicDataStoreFactory dataStoreFactory, IContentLoader contentLoader)
        {
            _dataStoreFactory = dataStoreFactory;
            _contentLoader = contentLoader;
        }

        public IndexRecentHits()
        {
            IsStoppable = true;
        }

        /// <summary>
        /// Called when a user clicks on Stop for a manually started job, or when ASP.NET shuts down.
        /// </summary>
        public override void Stop()
        {
            _stopSignaled = true;
        }

        /// <summary>
        /// Called when a scheduled job executes
        /// </summary>
        /// <returns>A status message to be stored in the database log and visible from admin mode</returns>
        public override string Execute()
        {
            //Call OnStatusChanged to periodically notify progress of job for manually started jobs
            OnStatusChanged(String.Format("Beginning processing of recent hits"));

            var totalProcessed = 0;
            var errorCount = 0;

            //Get the recent hit counts
            if (!int.TryParse(_timeWindow, out int recentHours))
            {
                recentHours = 24;
            }
            var fromDate = DateTime.Now.AddHours(0 - recentHours).ToUniversalTime().ToString("o");

            // Set up the request
            var request = GetTrackingRequest($"EventType eq epiPageView and EventTime gt {fromDate}", _resultsPerPage);

            // Gather the data from Profile Store
            ProcessEventResults(1, request);

            if (_stopSignaled)
            {
                return "Execution was cancelled by user";
            }


            var store = _dataStoreFactory.CreateStore(typeof(RecentHit));
            store.DeleteAll();

            foreach (var hit in _recentHits)
            {
                if (_stopSignaled)
                {
                    return "Execution was cancelled by user";
                }
                try
                {
                    var keyParts = hit.Key.Split('_');
                    var page = _contentLoader.Get<SitePageData>(new Guid(keyParts.FirstOrDefault() ?? Guid.Empty.ToString()));
                    var recentHit = new RecentHit
                    {
                        PageId = page.ContentLink.ID,
                        PageTypeId = page.ContentTypeID,
                        Parents = _contentLoader.GetAncestors(page.ContentLink).Select(x => x.ContentLink.ID).ToArray(),
                        Language = keyParts.LastOrDefault() ?? "en",
                        Hits = hit.Value
                    };
                    store.Save(recentHit);
                }
                catch (Exception)
                {
                    errorCount++;
                }
                totalProcessed++;
                if (totalProcessed.ToString().EndsWith("0"))
                {
                    OnStatusChanged($"Indexed {totalProcessed} of {_recentHits.Count} with {errorCount} errors");
                }
            }

            return $"Reindexed {totalProcessed} pages with {errorCount} errors";
        }

        #region Private Methods
        /// <summary>
        /// Makes a request to ProfileStore and processes results
        /// </summary>
        private void ProcessEventResults(int page, RestRequest request)
        {
            OnStatusChanged($"Fetching hits page {page}");
            if (_stopSignaled)
            {
                return;
            }

            //Handle pagination
            request.AddOrUpdateParameter("$skip", (page - 1) * _resultsPerPage);

            // Execute the request to get the events matching the filter
            var eventResponseObject = GetTrackingResponse(request);
            foreach (var result in eventResponseObject.Items)
            {
                //Add/update the hit count per event
                var key = $"{result.Payload.Epi.ContentGuid}_{result.Payload.Epi.Language}";
                if (_recentHits.ContainsKey(key))
                {
                    _recentHits[key]++;
                }
                else
                {
                    _recentHits.Add(key, 1);
                }
            }

            //Repeat until all pages of results have been processed
            if (eventResponseObject.Total > _resultsPerPage * page)
            {
                ProcessEventResults(page + 1, request);
            }

        }

        /// <summary>
        /// Builds the ProfileStore request
        /// </summary>
        private RestRequest GetTrackingRequest(string filter, int resultsPerPage)
        {
            var req = new RestRequest(_eventUrl, Method.GET);
            req.AddHeader("Ocp-Apim-Subscription-Key", _appKey);

            req.AddParameter("$top", resultsPerPage);
            req.AddParameter("$filter", filter);
            return req;
        }

        /// <summary>
        /// Serialises the ProfileStore response into an object
        /// </summary>
        private TrackingObjectResponse GetTrackingResponse(RestRequest request)
        {
            var client = new RestClient(_apiRootUrl);
            var getEventResponse = client.Execute(request);
            return JsonConvert.DeserializeObject<TrackingObjectResponse>(getEventResponse.Content);
        }
        #endregion

    }

Putting it all together

So, now we’ve collected the data and put it together into a handy list format, it’s time to put it to use and get the listing onto our site. To do this I’m going to create a couple of helper functions to get the data (one returning typed data, the other returning all data):

public IEnumerable<T> GetPopularPages<T>(ContentReference ancestor, string language, int numberOfResults) where T : PageData
{
    var store = _dataStoreFactory.CreateStore(typeof(RecentHit));
    var contentTypeId = _contentTypeRepository.Load<T>().ID;
    var hits = store.Items<RecentHit>().Where(x =>     x.Parents.Contains(ancestor.ID) && x.Language.Equals(language) && x.PageTypeId.Equals(contentTypeId)).OrderByDescending(x => x.Hits).Take(numberOfResults).ToList();
    var contentRefs = hits.Select(x => new ContentReference(x.PageId));
    return _contentLoader.GetItems(contentRefs, new LoaderOptions() { LanguageLoaderOption.Specific(CultureInfo.GetCultureInfo(language)) }).OfType<T>();
}

public IEnumerable<IContent> GetPopularPages(ContentReference ancestor, string language, int numberOfResults)
{
    var store = _dataStoreFactory.CreateStore(typeof(RecentHit));
    var hits = store.Items<RecentHit>().Where(x => x.Parents.Contains(ancestor.ID) && x.Language.Equals(language)).OrderByDescending(x => x.Hits).Take(numberOfResults);
    var contentRefs = hits.Select(x => new ContentReference(x.PageId));
    return _contentLoader.GetItems(contentRefs, new LoaderOptions() { LanguageLoaderOption.Specific(CultureInfo.GetCultureInfo(language)) });
}

From here, it’s just a matter calling one of these functions from a block or page and rendering the result. In my case, I created a block and added it to the news and events page of the alloy site which gave me this…

For those interested, I’ve added the code (including the block) to a Gist on GitHub but do bear in mind that this has been created as a proof-of-concept rather than a battle-hardened, production-ready feature so use it with caution.