Introducing Vulcan, the lightweight Elasticsearch client for Episerver

I like Episerver Find. No, more than that… I LOVE it. Fast, configurable, powerful, reliable, scalable… it’s everything you need in search and SO much more. I try and throw all my site data that I can into it, and I’ve never been sorry yet. If you are looking at an Episerver project, I’d suggest you just factor in Find right up front and make sure you budget for it (and, if you go Episerver Cloud, there’s a good chance you will get it bundled anyway.)

Having said all that, Episerver are fully aware that for much smaller sites that are licensed on-premise and trying to manage costs tightly, Find may be an expense too far. Even more than that, here in South Africa not only are costs very constrained (go see the value of the Rand for a clue) but Find isn’t available on a convenient node for Sub-Saharan Africa, and so latency is a little higher. For that reason, Episerver still provides Episerver Search as an option, and there are various other providers you can use, some of whom you’ll find in the Add-On Store. However, Episerver Search is somewhat limited (outdated port of Lucene.Net, limited capabilities when it comes to some features such as faceting, as well as scalability and UI is… well… non-existent) and the 3rd party options are, again, paid-for. So what does that leave us with?

This is the (admittedly small) niche where Vulcan could be useful. Firstly, what is it not; It’s NOT Episerver Find and it’s NOT supported – not by Episerver, by me or anyone else. If you want a proper, enterprise-level, supported product with fabulous UI and integration, go dig a little in your pockets and get Find. So what IS Vulcan? It is a small, lightweight wrapper around Elasticsearch’s NEST client that provides helpers and tools to index and search for CMS and Commerce content. As of now, there is no UI (other than an indexing scheduled job) and configuration is fairly limited. That said, it’s simple and as it’s Open Source, you can do what you like with it when it comes to extending and customising it. You can even host your own Elasticsearch instance, so it could be very cost effective! I just have one request – if you add a cool feature or fix a bug, be prepared to commit it back into the repo so that we can all benefit. Sound fair?

I’m also fairly inexperienced with Elasticsearch and NEST, and the documentation isn’t fabulous which doesn’t help, so if you think I’m barking up the wrong tree then I’m all ears. Maybe I’ve spectacularly missed the point or done things in a much harder way than necessary. So, how do you get Vulcan and use it? I’m hoping at some point to make it available as a Nuget package, but for now you can pull the latest code from here:

Vulcan repo on GitLab

Note that it’s a PRIVATE repo and you’ll need to login then request access. I’ve done this simply because it’s very much in alpha and I don’t want it grabbed by just anyone. I’d rather know where it’s going. Once you’ve downloaded the repo you’ll see two projects:

TcbInternetSolutions.Vulcan.Core – this is the main project including an implementation

TcbInternetSolutions.Vulcan.Commerce – this adds Commerce support

Compile them then drop the appropriate assemblies into your CMS/Commerce project’s bin folder (or just add the projects to your solution and reference them if you prefer). For CMS, you just need the Core one. For Commerce, you’ll need Core and Commerce. Note that I’ve built against the very latest version of Episerver CMS and Commerce. No reason for that really, and I’ll probably refactor to an older set of packages at some point, but it was just easiest at the time. Once that’s done there is just one last thing to do – get an Elasticsearch instance somewhere. I’ve found that FacetFlow is awesome… they give you fairly sizable index for free. Once you have registered and gotten yourself a URL, you need to plug that into the appSettings of your web.config along with whatever your index name should be (you can pick your own). For example:

<add key="VulcanUrl" value="https://yourkey:@somesite.azr.facetflow.io" />
<add key="VulcanIndex" value="vulcan" />

All of the Vulcan features are exposed via the IVulcanHandler interface, which is registered with the IoC container. You can therefore inject it, constructor it or service locate it… whatever flavour floats your boat. Once you have it, you can then use it like any other NEST client, but I have added some helper methods:

ISearchResponse<IContent> SearchContent<T>(Func<SearchDescriptor<T>, SearchDescriptor<T>> searchDescriptor = null) where T : class, IContent;

void IndexContent(IContent content);

void DeleteContent(IContent content);

var searchResponse = VulcanHandler.Service.Client.SearchContent<StandardPage>();

 

This will get all of the StandardPage content in the index. Vulcan does support inheritance, and so anything that inherits from StandardPage will also be returned. Normally you would then look at the Documents property of the response to get the content. You can still do that, but you’ll find that inside there you will have a set of Vulcan content constructs. This is done for performance… they inherit from IContent and you can use them directly if you like for the common properties, but probably you want the actual content hits. There are a couple of extension methods that I’ve created that can help you here. Simply add a ‘using’ to TcbInternetSolutions.Vulcan.Core.Extensions and then you can directly get the content by using the GetContents() extension method on the search response. This will give an IEnumerable of IContent.

 

var contents = VulcanHandler.Service.Client.SearchContent<StandardPage>().GetContents();

 

You may also want to get the content along with the ‘hits’ that correspond to them. Typically, this would be for something like hit highlighting. Here is a Razor snippet that shows the extension method GetHitContents() which returns an IDictionary of the IHit and the IContent (the model is passing the search response in the ContentHits property):

 

@foreach (var hit in Model.ContentHits.GetHitContents())
{
    <div class="listResult">
        <h3><a href="@Url.ContentUrl(hit.Value.ContentLink)">@hit.Value.Name</a></h3>
        @if (hit.Key.Highlights != null)
        {
            foreach (var highlight in hit.Key.Highlights)
            {
                <p>@Html.Raw(string.Join(",", highlight.Value.Highlights))</p>
            }
        }
        <hr />
    </div>
}

Now you’ve seen the basics, you can make it useful by using the standard NEST Query DSL and Aggregations to do funky searches, facets etc. For example:

 

model.ContentHits = VulcanHandler.Service.Client.SearchContent<IContent>(d => d.Query(query => query.SimpleQueryString(sq => sq.Query(q)))
                    .Highlight(h => h.Encoder("html").Fields(f => f.Field("*")))
                    .Aggregations(agg => agg.Terms("types", t => t.Field("_type"))));

This snippet passes in a simple text string q into the query, then hit-highlights on all fields (encoded HTML so that it’s neat) and facets the return on the type of content. Put it all together, and you get something like this (I threw together a little test in the Alloy templates):

 

So that’s Vulcan! A lightweight client side library to drive an Episerver website. And it’s Open Source. Which is nice. If you have questions, comments or queries let me know… post here or you can email me on firstname.lastname@episerver.com.

 

DISCLAIMER: This project is in no way connected with or endorsed by Episerver. It is being created under the auspices of a South African company and is entirely separate to what I do as an Episerver employee.