Views: 1040
Number of votes: 7
Average rating:

Using content webhooks with Episerver

The concept of webhooks isn't a new one but in recent years it has become a core component to many headless and SaaS implementations. Without direct access to the underlying data, solutions built in this way typically rely on HTTP APIs to access data and expose functionality, but relying on HTTP APIs alone can result in unnecessary overheads. Excessive data transfer and the network latency introduced can, if not carefully managed, make applications feel clunky and unresponsive. That’s where webhooks come in. 

At a high level, webhooks offer a basic pub/sub pattern implemented using standard HTTP requests. A URL is registered with the source system and when certain events occur in that system, an HTTP request is made to the URL containing information on the event which occurred. The data sent in the http request can then be processed by the receiving system, allowing for scenarios where data is refreshed in response to changes to that data rather than having to periodically check.

While headless content delivery may be a key use case for web hooks, it’s certainly not the only use case. Webhooks can be of use in a whole range of scenarios, particularly when combined with low-code integration platforms such as Azure Logic Apps, IFTTT and Zapier. These platforms provide ready built integrations to an extensive catalogue of services which you can put together like lego blocks enabling you to build complex integrations and data processing tasks without the need to write or deploy any code, and all of them can be triggered by webhooks.

As great as this may sound, there's something missing. While Episerver / Optimizely does have some webhook functionality (most notably in Forms though also in Campaign), there is no functionality to trigger webhooks from content events. I can understand why this isn’t provided out-of-the-box, after all, we can programmatically attach handlers to any Episerver events and perform whatever actions we need. But codebase updates and deployments introduce their own inefficiencies and, with decoupled architectures becoming more common, webhooks offer a simpler, more unified solution. Sounds like a job for an add-on...

Introducing Kin and Carta Webhooks

At February’s Developer Happy Hour I demonstrated an add-on I’ve been working on which adds webhook functionality to Episerver / Optimizely CMS solutions and I’m pleased to announce that it is now available via the Episerver NuGet feed. After installing the add-on, an additional “webhooks” menu item is added to the CMS navigation for users with admin access which links to the webhook UI. The UI allows for webhooks to be created, tested, updated and deleted as well as allowing for an at-a-glance view of the last result for each webhook. Clicking on the last result pops up the full response received, allowing for easier debugging.

Creating a webhook

To set up a new webhook, click the “New Webhook” button and enter a name to identify the webhook and a URL which will be called when the webhook is fired, then select a parent node which can be used to limit triggering the webhook to only the selected node or its children.

You should also select one or more content events which will determine when the web hook is triggered. Out of the box, the add on supports 5 content events:

  • Publish - Called once a content item has been published
  • Moving - Called before a content item has been moved
  • Move - Called after a content item has been moved
  • Recycled - Called after a content item  has been moved to the recycle bin
  • Deleted - Called before a content item has been permanently deleted

N.B. Content save events have been intentionally ignored as they occur almost constantly while editing content however it is possible to register your own events if this is required. I’ll cover how to do that in a subsequent post.

You have the option to select one or more content types so that the webhook will trigger only on events involving content of the selected types. Leaving this blank defaults to triggering on all content types (assuming they’re in the correct part of the tree). Should you require, you can also add up to 5 custom HTTP headers which could be used for sending authentication information, keys or any other additional information your receiver may require in the header.

Once all of the relevant information has been entered, click save to save the webhook and return to the webhook overview screen. You can now click on the “Test” button to trigger the webhook as though a content event had occurred. To do this, the add-on finds the first content item which matches the criteria set and triggers the webhook using that content item. The result of the test will appear in a popup.

Payload data

When one of the supported events is triggered by a content event, any webhooks targeted to that event, content location and content type will be retrieved and a POST request will be made to the defined URL. This request is made on a background thread to avoid impacting the default processing of the action which triggered the event. The body of the post request contains information about the event fired, the content that triggered it and any other content items impacted by the event in JSON format. The top level JSON properties are:

  • Content - a JSON representation of the content which triggered the event. This uses the content delivery API to serialise the content and so the structure of the serialised content data should match data retrieved from the API, including customisations such as flattening the structure.
  • ContentInfo - high level information about the content (Name, IDs, Language, URL)
  • ReferencedBy - information about content items (Name, IDs, Language, URL) which reference the content which triggered the event. This can be useful when using webhooks to invalidate remote caches.
  • EventType - the name of the event which occurred
  • EventTime - the date/time at which the event occurred

There is also a field called ExtraData, though this exists for extension purposes and so will currently be null. More on this in a future post.

An easy way to explore the structure of the payload is to set up a webhook pointed to a service like webhook.site. To try this out, simply visit https://webhook.site/ and you will be assigned a temporary URL. You can then set up a webhook in Episerver using that URL and send a test. Any requests sent to that URL will appear as they are received and can be inspected to view the payload and header data.

Token Replacement

When creating a webhook, tokens can be used within the URL and custom header fields to allow information such as keys or environment-specific values to be pulled in. Tokens are in the format ${TokenName} where TokenName is a key registered using the RegisterPlaceholder(string token, string value) method of the IWebhookRepository. The value associated with the token will replace the token placeholder. In the event that the token name doesn't match a key, the token placeholder will be left as-is. 

Registering tokens and their values will typically be done in an IInitializableModule though, if you want to register all appsettings values from your web.config file, you can add the following app setting to your config:
<add key="WH:AutoRegisterAppSettings" value="true" /> 

N.B. Before using this config value, ensure you are aware of the implications. By registering all appsettings like this, it is possible to extract potentially sensitive data from configuration and send it to an arbitrary URL.

Further information

During the February Dev Happy Hour I demonstrated a few of the ways the add-on could be used, including a demo of how we can use webhooks with Azure Logic Apps to selectively invalidate a CDN cache when cached content is updated. You can find the recording on the EMEA Dev Happy Hour page (the webhooks bit starts at the 30 minute mark).

The add-on is available in the Episerver NuGet feed under “KinAndCarta.Webhooks” and requires no particular set-up other than installing the package. Full source code is also available on GitHub under an MIT license.

I hope you find this add-on useful and do let me know in the comments if you’ve got any feedback, questions, comments or suggestions.

Mar 31, 2021

MartinOttosen
( By MartinOttosen, 3/31/2021 10:03:49 AM)

This looks great Paul!

Allan Thraen
( By Allan Thraen, 3/31/2021 10:06:11 AM)

Thank you, Paul! I've had the same idea in the back of my head for the last 10 years, but never could I pull myself together to actually package it up. So well done, looks like a neat implementation and I'll definitely be using this in the future!

Scott Reed
( By Scott Reed, 3/31/2021 10:39:13 AM)

Sweet!!!

David Knipe
( By David Knipe, 3/31/2021 10:40:11 AM)

Great stuff Paul, looks awesome and well thought through!

One question is you mention "Publish - Called once a page has been published" but assume it works on anything in the content repo? Certainly looks like it works on blocks from the screen shots :) 

Jay Wilkinson
( By Jay Wilkinson, 3/31/2021 10:56:11 AM)

Nice work Paul

Scott Reed
( By Scott Reed, 3/31/2021 11:00:12 AM)

The only suggestion I have is I've worked on a lot of sites with lots and lots of content types. It might be nice to have a little filter box and some checkboxes (page, blocks, media) just to narrow the list of content types down, when you've got over 100 any of the admin UI for selecting them is a nightmare.

Paul Gruffydd
( By Paul Gruffydd, 3/31/2021 12:48:25 PM)

Thanks all for the enthusiastic comments.

@Allan - It's an idea I've had for a few years too, it's just taken a while for me to find the time to actually do it. It was sat at the PoC stage, untouched for the best part of a year.

@David - Well spotted, thanks. Yes, it works on any IContent so that's pages, blocks, media or even virtual templates! I'll update it to say "content" rather than "page".

@Scott - Yep, I'd agree. I installed it on a foundation site when I first built the NuGet package to test on a varety of sites and the list of types was fairly daunting. It's on my to-do list to make that list a bit more manageable. There are a few UI related things to tidy up. That's one of them as is making the text translatable as, at present, most of the text shown in the add-on is only in English. For now though, I've focussed on getting the first release out to see whether it's actually something people would find useful. If it is then I've got lots of ideas for enhancements.

KennyG
( By KennyG, 3/31/2021 10:14:13 PM)

This is an awesome idea!

Sanjay Kumar
( By Sanjay Kumar, 4/1/2021 11:31:32 AM)

Nice Article!

Stein-Viggo Grenersen
( By Stein-Viggo Grenersen, 4/8/2021 3:17:14 PM)

Really cool and useful module! Together with Zapier it provides a lot of interesting use cases. For one, we have set it up to post messages to social channels like Facebook on our demo environment via Zapier.

Surjit Bharath
( By Surjit Bharath, 4/8/2021 6:33:03 PM)

This is fantastic, I can think of a few use cases for this!

Stein-Viggo Grenersen
( By Stein-Viggo Grenersen, 4/9/2021 8:22:20 AM)

PS. If you set it up to post eg. to Facebook be sure to filter on language as every language version of a page will get posted as a separate post.

K Khan
( By K Khan, 4/9/2021 8:56:36 AM)

@Paul, awesome, fantastic, great work, you really did it :) I am also one of them who have been thinking this since 2016 :D https://world.episerver.com/blogs/K-Khan-/Dates/2016/3/zap-integration/

Jay Wilkinson
( By Jay Wilkinson, 4/9/2021 9:22:40 AM)

This is indeed really cool, and has lots of potential for expanding the scope. 

An idea that springs to mind is the Scheduled Jobs pipeline in Episerver. I've seen many requirements for an Email confirmation of a scheduled job being completed - or more commonly, an Email alert for when it fails.

Doesn't look like there are events to hook into for Scheduled Jobs though - which brings a challenge, but the status of scheduled jobs is recorded in the tblScheduledItemLog table, so definitely a possibility! 

Paul Gruffydd
( By Paul Gruffydd, 4/9/2021 11:45:22 AM)

Thanks all.

@Stein-Viggo Grenersen - Nice. That's similar to the example I use to explain a basic use case (except I've used twitter rather than FB). As you spotted, each time a language version is published, that's a publish in its own right so you'll end up with the webhook firing on each publish of each language. I did consider including a filter for language but it's only really relevant to publish events so I've left it to the receiver to filter. Beyond demos, if you were using it to post to social media, you'd probably want to do something to ensure it only posts on first publish rather than when updates are published. You could do that by keeping a list of what's been published and checking in your receiver, or you could register a custom webhook event which fires only when a content item is first published. I'm aiming to cover the custom event approach in my next post.

@K Khan - Interesting. I'd not seen that post but I do remember attending your talk on using Azure Logic Apps with commerce at Ascend a few years ago and thinking how simple and powerful a process it could be with the use of webhooks. For the record, commerce support is the next major feature I'm looking at.

@Jay Wilkinson - That's an interesting idea. I've not tried hooking into scheduled job events but, from what I can see, IScheduledJobEvents has an "Executed" event which you could use to register a custom webhook event. The webhooks in this add-on are intended for events relating to IContent instances so need a reference to an IContent in the payload but you could use the root node for that and pass through information on the scheduled job and its status using the "ExtraData" property.

Michal Toman
( By Michal Toman, 4/22/2021 11:52:22 AM)

Working perfectly with Zapier! Thanks!

Please login to comment.