The Next Step for Episerver CMS Labs - User Telemetry in BlockEnhancements

As of EPiServer.Labs.BlockEnhancements 0.7.0 it will support sending some telemetry data to help the CMS UI team determine which of the many great improvements should be fully supported, with all the complicated combinations of Languages, Content Approval, Project Mode, and more. This telemetry technology and how we work with it in CMS UI will move into the main CMS UI package in the near future. Opting-in on self-hosted instances is optional. DXP hosted instances will have this enabled.

Help us do this right. We want to learn about how the CMS UI is used, while respecting our users privacy. See the Telemetry Information section and the code on Github. Create issues there with your concerns or post here on this blog post. If you prefer to write in private you can reach me at jp.johansson@episerver.com

As you might have read in Introducing Episerver CMS Labs there are certain improvements that are just too big to do right away without knowing if we're going to hit the mark. We introduced Labs to validate potential features before implementing them fully and we're very happy with the feedback we've received from our first Labs: BlockEnhancements. I have personally heard from many Sales Architects, Trainers, PMs, and EMVP's; how it has completely changed the experience of working with blocks. That's wonderful!

You might also notice that I didn't mention Editors or Marketers; the ones that work all day in our UI, the ones we build all of this for. Which of the six main features are making them happy? All of them? One of them? None of them? Does the inline publish from the Inline Block Edit (v0.5.0) fit into a workflow of "edit, then publish" in such a way that the Inline Publish from the Assets Pane (v0.1.0) is no longer needed? Does Smart Publish (v0.2.0) work so well that Inline Publish from the Content Area (v0.1.0) is no longer used? And does anyone ever try to publish anything less than "everything" when the Smart Publish dialog shows up for configuration? It's really hard to tell.

Until now. With EPiServer.Labs.BlockEnhancements 0.7.0 we will finally get some of those answers. It will enable us to use AppInsights to track feature usage and give a high level answer to some of our questions, as you can see here:

(This is how the data looks after CMS UI and QA has verified the UI functionality for this release)

We plan to fully support one of the main publishing paths from BlockEnhancements, and we need some data to see which one. Perhaps we learn that despite the positive remarks neither really hit the mark and we need to do more Labs.

Privacy

The CMS UI team is very interested in what our users use and do not use. What should we improve, what should we remove, what ... So many questions. 😄 But just gathering data and then trying to find questions that can be answered with the data is perhaps more fun than useful. Instead, we will start with a question and only add the minimum of tracking events to answer that question in a reasonable way.

We know from meeting users that they can have different preferences due to previous experiences. We also know from experience that sometimes a feature is used very differently from what we intended due to modifications on the partner/client end. To be able to filter data to determine outliers we will use anonymized id's for users and clients.

We are mainly interested in aggregated data, which you can see in our dashboard above. We will enable this on DXP environments, but if you're self hosted you can enable Telemetry with the configuration:

public void ConfigureContainer(ServiceConfigurationContext context)
{
    context.Services.Configure<TelemetryOptions>(options => options.OptedIn = true);
}

We're trying to be as transparent as possible so please take a look at in the Telemetry Information section where all the tracked events are carefully detailed.

We're looking at how we can share this data with you and how it can be useful for add-ons, both 1st and 3rd party ones. We're taking it slow to make sure we do this right and respect everyone's concerns around privacy. We will start with something simpler, perhaps some blog posts from the team with what we learned - would that be of interest to you?

This is just a first step into being a bit more data-driven in our UX decisions. We plan to include this technology in the main CMS UI product. While this is in BlockEnhancements you can see every tracked event detailed in the Readme, and see it in the code as well. What we track in CMS UI will be of the same level of detail, but won't be as easy to be as transparent with. Help us do it right in BlockEnhancements so that we're all happy and confident with this in CMS UI. We tried to make it as clear as possible for you to review the code in this Github pull request. Create issues there with your concerns or post here on this blog post. If you prefer to write in private you can reach me at jp.johansson@episerver.com