Paul Gruffydd
Mar 31, 2021
  2065
(7 votes)

Using content webhooks with Episerver

The concept of webhooks isn't a new one but in recent years it has become a core component to many headless and SaaS implementations. Without direct access to the underlying data, solutions built in this way typically rely on HTTP APIs to access data and expose functionality, but relying on HTTP APIs alone can result in unnecessary overheads. Excessive data transfer and the network latency introduced can, if not carefully managed, make applications feel clunky and unresponsive. That’s where webhooks come in. 

At a high level, webhooks offer a basic pub/sub pattern implemented using standard HTTP requests. A URL is registered with the source system and when certain events occur in that system, an HTTP request is made to the URL containing information on the event which occurred. The data sent in the http request can then be processed by the receiving system, allowing for scenarios where data is refreshed in response to changes to that data rather than having to periodically check.

While headless content delivery may be a key use case for web hooks, it’s certainly not the only use case. Webhooks can be of use in a whole range of scenarios, particularly when combined with low-code integration platforms such as Azure Logic Apps, IFTTT and Zapier. These platforms provide ready built integrations to an extensive catalogue of services which you can put together like lego blocks enabling you to build complex integrations and data processing tasks without the need to write or deploy any code, and all of them can be triggered by webhooks.

As great as this may sound, there's something missing. While Episerver / Optimizely does have some webhook functionality (most notably in Forms though also in Campaign), there is no functionality to trigger webhooks from content events. I can understand why this isn’t provided out-of-the-box, after all, we can programmatically attach handlers to any Episerver events and perform whatever actions we need. But codebase updates and deployments introduce their own inefficiencies and, with decoupled architectures becoming more common, webhooks offer a simpler, more unified solution. Sounds like a job for an add-on...

Introducing Kin and Carta Webhooks

At February’s Developer Happy Hour I demonstrated an add-on I’ve been working on which adds webhook functionality to Episerver / Optimizely CMS solutions and I’m pleased to announce that it is now available via the Episerver NuGet feed. After installing the add-on, an additional “webhooks” menu item is added to the CMS navigation for users with admin access which links to the webhook UI. The UI allows for webhooks to be created, tested, updated and deleted as well as allowing for an at-a-glance view of the last result for each webhook. Clicking on the last result pops up the full response received, allowing for easier debugging.

Creating a webhook

To set up a new webhook, click the “New Webhook” button and enter a name to identify the webhook and a URL which will be called when the webhook is fired, then select a parent node which can be used to limit triggering the webhook to only the selected node or its children.

You should also select one or more content events which will determine when the web hook is triggered. Out of the box, the add on supports 5 content events:

  • Publish - Called once a content item has been published
  • Moving - Called before a content item has been moved
  • Move - Called after a content item has been moved
  • Recycled - Called after a content item  has been moved to the recycle bin
  • Deleted - Called before a content item has been permanently deleted

N.B. Content save events have been intentionally ignored as they occur almost constantly while editing content however it is possible to register your own events if this is required. I’ll cover how to do that in a subsequent post.

You have the option to select one or more content types so that the webhook will trigger only on events involving content of the selected types. Leaving this blank defaults to triggering on all content types (assuming they’re in the correct part of the tree). Should you require, you can also add up to 5 custom HTTP headers which could be used for sending authentication information, keys or any other additional information your receiver may require in the header.

Once all of the relevant information has been entered, click save to save the webhook and return to the webhook overview screen. You can now click on the “Test” button to trigger the webhook as though a content event had occurred. To do this, the add-on finds the first content item which matches the criteria set and triggers the webhook using that content item. The result of the test will appear in a popup.

Payload data

When one of the supported events is triggered by a content event, any webhooks targeted to that event, content location and content type will be retrieved and a POST request will be made to the defined URL. This request is made on a background thread to avoid impacting the default processing of the action which triggered the event. The body of the post request contains information about the event fired, the content that triggered it and any other content items impacted by the event in JSON format. The top level JSON properties are:

  • Content - a JSON representation of the content which triggered the event. This uses the content delivery API to serialise the content and so the structure of the serialised content data should match data retrieved from the API, including customisations such as flattening the structure.
  • ContentInfo - high level information about the content (Name, IDs, Language, URL)
  • ReferencedBy - information about content items (Name, IDs, Language, URL) which reference the content which triggered the event. This can be useful when using webhooks to invalidate remote caches.
  • EventType - the name of the event which occurred
  • EventTime - the date/time at which the event occurred

There is also a field called ExtraData, though this exists for extension purposes and so will currently be null. More on this in a future post.

An easy way to explore the structure of the payload is to set up a webhook pointed to a service like webhook.site. To try this out, simply visit https://webhook.site/ and you will be assigned a temporary URL. You can then set up a webhook in Episerver using that URL and send a test. Any requests sent to that URL will appear as they are received and can be inspected to view the payload and header data.

Token Replacement

When creating a webhook, tokens can be used within the URL and custom header fields to allow information such as keys or environment-specific values to be pulled in. Tokens are in the format ${TokenName} where TokenName is a key registered using the RegisterPlaceholder(string token, string value) method of the IWebhookRepository. The value associated with the token will replace the token placeholder. In the event that the token name doesn't match a key, the token placeholder will be left as-is. 

Registering tokens and their values will typically be done in an IInitializableModule though, if you want to register all appsettings values from your web.config file, you can add the following app setting to your config:
<add key="WH:AutoRegisterAppSettings" value="true" /> 

N.B. Before using this config value, ensure you are aware of the implications. By registering all appsettings like this, it is possible to extract potentially sensitive data from configuration and send it to an arbitrary URL.

Further information

During the February Dev Happy Hour I demonstrated a few of the ways the add-on could be used, including a demo of how we can use webhooks with Azure Logic Apps to selectively invalidate a CDN cache when cached content is updated. You can find the recording on the EMEA Dev Happy Hour page (the webhooks bit starts at the 30 minute mark).

The add-on is available in the Episerver NuGet feed under “KinAndCarta.Webhooks” and requires no particular set-up other than installing the package. Full source code is also available on GitHub under an MIT license.

I hope you find this add-on useful and do let me know in the comments if you’ve got any feedback, questions, comments or suggestions.

Mar 31, 2021

Comments

MartinOttosen
MartinOttosen Mar 31, 2021 10:03 AM

This looks great Paul!

Allan Thraen
Allan Thraen Mar 31, 2021 10:06 AM

Thank you, Paul! I've had the same idea in the back of my head for the last 10 years, but never could I pull myself together to actually package it up. So well done, looks like a neat implementation and I'll definitely be using this in the future!

Scott Reed
Scott Reed Mar 31, 2021 10:39 AM

Sweet!!!

David Knipe
David Knipe Mar 31, 2021 10:40 AM

Great stuff Paul, looks awesome and well thought through!

One question is you mention "Publish - Called once a page has been published" but assume it works on anything in the content repo? Certainly looks like it works on blocks from the screen shots :) 

James Wilkinson
James Wilkinson Mar 31, 2021 10:56 AM

Nice work Paul

Scott Reed
Scott Reed Mar 31, 2021 11:00 AM

The only suggestion I have is I've worked on a lot of sites with lots and lots of content types. It might be nice to have a little filter box and some checkboxes (page, blocks, media) just to narrow the list of content types down, when you've got over 100 any of the admin UI for selecting them is a nightmare.

Paul Gruffydd
Paul Gruffydd Mar 31, 2021 12:48 PM

Thanks all for the enthusiastic comments.

@Allan - It's an idea I've had for a few years too, it's just taken a while for me to find the time to actually do it. It was sat at the PoC stage, untouched for the best part of a year.

@David - Well spotted, thanks. Yes, it works on any IContent so that's pages, blocks, media or even virtual templates! I'll update it to say "content" rather than "page".

@Scott - Yep, I'd agree. I installed it on a foundation site when I first built the NuGet package to test on a varety of sites and the list of types was fairly daunting. It's on my to-do list to make that list a bit more manageable. There are a few UI related things to tidy up. That's one of them as is making the text translatable as, at present, most of the text shown in the add-on is only in English. For now though, I've focussed on getting the first release out to see whether it's actually something people would find useful. If it is then I've got lots of ideas for enhancements.

KennyG
KennyG Mar 31, 2021 10:14 PM

This is an awesome idea!

Sanjay Kumar
Sanjay Kumar Apr 1, 2021 11:31 AM

Nice Article!

Stein-Viggo Grenersen
Stein-Viggo Grenersen Apr 8, 2021 03:17 PM

Really cool and useful module! Together with Zapier it provides a lot of interesting use cases. For one, we have set it up to post messages to social channels like Facebook on our demo environment via Zapier.

Surjit Bharath
Surjit Bharath Apr 8, 2021 06:33 PM

This is fantastic, I can think of a few use cases for this!

Stein-Viggo Grenersen
Stein-Viggo Grenersen Apr 9, 2021 08:22 AM

PS. If you set it up to post eg. to Facebook be sure to filter on language as every language version of a page will get posted as a separate post.

K Khan
K Khan Apr 9, 2021 08:56 AM

@Paul, awesome, fantastic, great work, you really did it :) I am also one of them who have been thinking this since 2016 :D https://world.episerver.com/blogs/K-Khan-/Dates/2016/3/zap-integration/

James Wilkinson
James Wilkinson Apr 9, 2021 09:22 AM

This is indeed really cool, and has lots of potential for expanding the scope. 

An idea that springs to mind is the Scheduled Jobs pipeline in Episerver. I've seen many requirements for an Email confirmation of a scheduled job being completed - or more commonly, an Email alert for when it fails.

Doesn't look like there are events to hook into for Scheduled Jobs though - which brings a challenge, but the status of scheduled jobs is recorded in the tblScheduledItemLog table, so definitely a possibility! 

Paul Gruffydd
Paul Gruffydd Apr 9, 2021 11:45 AM

Thanks all.

@Stein-Viggo Grenersen - Nice. That's similar to the example I use to explain a basic use case (except I've used twitter rather than FB). As you spotted, each time a language version is published, that's a publish in its own right so you'll end up with the webhook firing on each publish of each language. I did consider including a filter for language but it's only really relevant to publish events so I've left it to the receiver to filter. Beyond demos, if you were using it to post to social media, you'd probably want to do something to ensure it only posts on first publish rather than when updates are published. You could do that by keeping a list of what's been published and checking in your receiver, or you could register a custom webhook event which fires only when a content item is first published. I'm aiming to cover the custom event approach in my next post.

@K Khan - Interesting. I'd not seen that post but I do remember attending your talk on using Azure Logic Apps with commerce at Ascend a few years ago and thinking how simple and powerful a process it could be with the use of webhooks. For the record, commerce support is the next major feature I'm looking at.

@Jay Wilkinson - That's an interesting idea. I've not tried hooking into scheduled job events but, from what I can see, IScheduledJobEvents has an "Executed" event which you could use to register a custom webhook event. The webhooks in this add-on are intended for events relating to IContent instances so need a reference to an IContent in the payload but you could use the root node for that and pass through information on the scheduled job and its status using the "ExtraData" property.

Michal Toman
Michal Toman Apr 22, 2021 11:52 AM

Working perfectly with Zapier! Thanks!

Please login to comment.
Latest blogs
Plug-in manager is back in CMS 12

Plug-in manager is back in the UI, what is it and how can i use it?

Luc Gosso (MVP) | Oct 6, 2022 | Syndicated blog

Display Child Pages in Content Delivery API Response

The below example will implement an instance of IContentConverterProvider to customise the serialisation of PageData and output child pages in the...

Minesh Shah (Netcel) | Oct 4, 2022

Bring the Report Center back in Optimizely CMS 12

The Report Center has been a part of Optimizely CMS since its first debut in version 5R2 in 2008, but in CMS 12, it's removed! Don't despair! Make...

Tomas Hensrud Gulla | Oct 4, 2022 | Syndicated blog

Customizing Property Lists in Optimizely CMS

Generic property lists is a cool editorial feature that has gained a lot of popularity - in spite of still being unsupported (officially). But if y...

Allan Thraen | Oct 2, 2022 | Syndicated blog

Optimizely names Luminary Senior Developer, Ynze Nunnink, OMVP

Luminary Senior Developer and Optimizely Lead, Ynze Nunnink has secured the coveted position of Optimizely MVP. Earning a Platinum badge for his...

Ynze | Oct 2, 2022 | Syndicated blog

Content Delivery API – The Case of the Duplicate API Refresh Token

Creating a custom refresh provider to resolve the issues with duplicate tokens in the DXC The post Content Delivery API – The Case of the Duplicate...

David Lewis | Sep 29, 2022 | Syndicated blog