Blog posts by Mark Stott

Using Caching and Async to Improve Performance

Tue, 14 Apr 2026 06:51:16 GMT

Almost every CMS served website has some concept of dynamic content resolution. A means to serve up related or latest content links while minimising the burden on the content editors. This style of functionality is commonly delivered through functionality such as Search & Navigation, Content Graph, Algolia or some other indexing functionality. Another common use of dynamic content resolution are mega menus where the structure is based on the IA of the site and uses some form of recursive logic. For an In-Process or Traditional CMS, these can have a significant impact on the server response time for a page.

📖Note: The server response time is the time between the server receiving a request and delivering it’s response, disregarding any transmission time between the server and the client.

In the following sections I’m going to be talking about techniques I have used in practice on a high traffic website that was observed to be having performance issues such as Thread Pool Starvation.

What is Thread Pool Starvation?

Think of your application like a restaurant with a fixed number of chefs. Each chef (thread) can handle one order (request) at a time. Most of the time, orders are completed quickly and everything flows. But sometimes a chef needs extra ingredients, so they stop and wait.

Now imagine multiple chefs all waiting at the same time. They’re not cooking, but they’re also not free to take new orders. If enough chefs are waiting, no new meals get made. Orders pile up. The kitchen slows to a crawl.

That’s Thread Pool Starvation. In a .NET application, threads handle incoming requests. When those threads are blocked, typically waiting on I/O like database calls or search queries, they can’t be reused. Under load, this leads to:

Requests queuing up
Slower response times
An unresponsive application

Importantly, this isn’t a resource problem. CPU and memory might be fine, but your threads are tied up doing nothing. This is why synchronous calls are dangerous at scale. Each one blocks a thread until it completes. Asynchronous code avoids this by freeing the thread while waiting. Instead of standing idle, the “chef” moves on to the next order, keeping the system responsive and throughput high.

How does asynchronous code improve performance?

The number one culprit in Thread Pool Starvation was the use of the legacy synchronous methods for Search & Navigation. The asynchronous replacement for GetContentResult was introduced in EPiServer.Find 16.3.0 as GetContentResultAsync.

In my case, I had a listing block that used Optimizely Search & Navigation to return the 4 most recent news articles. On the surface, this seems like a lightweight query, but under load, even small synchronous calls add up quickly and contribute to thread pool starvation. The obvious first step was to switch from GetContentResult to GetContentResultAsync. However, simply calling the async method isn’t enough to realise the benefits. If you block on it further up the call stack (for example with .Result or .GetAwaiter().GetResult()), you’re back to the same problem, tying up threads while waiting.

To make this change effective, the async pattern needs to flow all the way back to the rendering layer. In Optimizely CMS, that meant pushing the change up to the ViewComponent level.

Instead of using:

public class ListingBlockComponent : BlockComponent<ListingBlock>
{
    protected override IViewComponentResult InvokeComponent(ListingBlock currentContent)
    {
        var results = await _searchClient
            .Search<ArticlePage>()
            .OrderByDescending(x => x.StartPublish)
            .Take(4)
            .GetContentResult();

        return View(results);
    }
}

The component was refactored to:

public class ListingBlockComponent : AsyncBlockComponent<ListingBlock>
{
    protected override async Task<IViewComponentResult> InvokeComponentAsync(ListingBlock currentContent)
    {
        var results = await _searchClient
            .Search<ArticlePage>()
            .OrderByDescending(x => x.StartPublish)
            .Take(4)
            .GetContentResultAsync();

        return View(results);
    }
}

There are two key changes here:

Inheriting from AsyncBlockComponent<T> instead of BlockComponent<T>
Switching to InvokeComponentAsync and fully embracing async/await

This allows the thread handling the request to be released while waiting for the search query to complete. Once the results are ready, execution resumes and the response is rendered.

The important point is that async isn’t just about swapping one method for another, it’s about ensuring the entire execution path is non-blocking. By pushing async all the way up to the ViewComponent, the application can handle significantly more concurrent requests without exhausting the thread pool.

How does adding caching improve performance?

Once asynchronous code removes unnecessary thread blocking, the next step is reducing how often you need to do the work in the first place. That’s where caching comes in.

In the case of a listing block, building the model can involve multiple operations, querying an index, resolving content references, applying enrichment such as category information, and shaping the final view model. While Search & Navigation code includes a level of caching already, the default duration is short and the cache only affects the results of the query and not the actions you perform on that data. Even when each individual operation is relatively fast, the combined cost adds up quickly under load. By caching the final model, you avoid repeating all of that work on every request.

The following code is an evolution of the previous example of a listing block, but this time following a standard pattern for adding caching into View Components which can be described as such:

Attempt to retrieve the data from the cache
Build model if it does not exist in the cache
Render the model

public class ListingBlockComponent(
  ISynchronizedObjectInstanceCache cache,
  IPageRouteHelper pageRouteHelper) : AsyncBlockComponent<ListingBlock>
{
    protected override async Task<IViewComponentResult> InvokeComponentAsync(ListingBlock currentContent)
    {
        var cacheKey = GetCacheKey(currentContent);
        if (!cache.TryGet<ListingBlockModel>(cacheKey, ReadStrategy.Immediate, out model))
        {
            model = BuildModel(currentContent);

            // The use of a master key here allows us to remove all our custom caches based on a constant known value.
            var evictionPolicy = new CacheEvictionPolicy(
              AppConstants.CacheDuration,
              CacheTimeoutType.Absolute, 
              [], [AppConstants.MasterKey]);
            cache.Insert(cacheKey, model, evictionPolicy)
        }

        return View(results);
    }

    private async ListingBlockModel BuildModel(ListingBlock currentContent)
    {
        // Model building logic (e.g. async Search & Navigation call)
    }

    private string GetCacheKey(ListingBlock currentContent)
    {
        var contentId = pageRouteHelper.ContentLink?.ID ?? 0;
        var language = pageRouteHelper.LanguageID;
        var blockId = (currentContent as IContent)?.ContentLink?.ID ?? contentId;

        // Create a new cache key unique to the block, page and language
        // e.g. MB:123:456:EN
        return $"MB:{contentId}:{blockId}:{language}";
    }
}

💡Top Tip: Short cache key names can result in faster cache lookups, but make sure you maintain the right level of uniqueness for your own usage.

Why Asynchronous Code and Caching work well together

This approach is effective because it combines both strategies:

Async ensures threads aren’t blocked while building the model
Caching ensures the model rarely needs to be rebuilt at all

The result is a dramatic reduction in both thread usage and external calls, exactly what you need to avoid thread pool starvation and improve response times under load.

In my scenario, caching wasn’t just applied in one place, it was introduced across several high-impact components that were rebuilt on every request. These included:

A mega menu driven by content hierarchy
Listing and Search pages
Listing Blocks (as described above)

The mega menu was the biggest offender. Each request triggered a large number of content lookups as the hierarchy was recursively resolved using IContentLoader. By caching the final menu model, those lookups were eliminated, reducing the number of cache and data operations by 500+ per request.

For the Listing and Search pages, the controllers were updated to use asynchronous controllers and to use the asynchronous Search and Navigation queries.

For listing blocks, caching removed repeated calls to the index and avoided rebuilding the model entirely. This ensured that common queries like “latest articles” were served near instantly while remaining unique to the page and language.

The Result

The combined use of caching and asynchronous coding resulted in a significant reduction in content lookups, index calls and less work per request. The Thread Pool Starvation issue was entirely eliminated and the average server response time for content pages was reduced to 0.045s per page over a 7 day period. Ultimately leading to a more stable environment that performed better with no loss of functionality.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12 & 13. You can find all of my content collated on https://www.stott.pro/

Stott Security v7 for Optimizely CMS 13

Thu, 09 Apr 2026 08:17:03 GMT

Stott Security version 7 is now available for Optimizely PaaS CMS 13, introducing support for configuring security headers by both application and host. This aligns the add-on more closely with how Optimizely CMS structures content and delivery, enabling far more precise and maintainable security policies.

Multiple Application and Host Support

In version 5 and below, Stott Security supported a global Content Security Policy with optional page-level extensions. In practice, this often meant combining requirements for both front-end delivery and the CMS editor experience into a single policy, leading to more permissive and larger configurations.

Version 7 introduces support for Global, Application, and Host-level configuration, allowing the Content Security Policy, Permissions Policy and Response Headers to be scoped more naturally to multi-channel and multi-site platforms.

To support this, a new Context Switcher has been introduced across the add-on UI. This allows users to quickly switch between Global, Application, and Host contexts when managing headers, making it clear exactly where a configuration is being applied.

Content Security Policy

Content Security Policy management in Stott Security is designed to minimise the need for developer intervention. The Settings and Sandbox sections can be configured globally and then overridden at Application or Host level, allowing different environments such as the frontend and CMS to operate under different modes (for example, enforcing vs report-only).

The real magic happens around the Sources. A flexible Content Security Policy (CSP) model isn’t just about defining what’s allowed, it’s about where those rules apply. By structuring CSP across multiple levels, you gain the ability to enforce strong defaults while still accommodating the specific needs of different parts of your platform.

In this model, the sources for the Content Security Policy are applied across four levels of scope and merged to create the final output specific to the page being served. Each level becomes progressively more specific, allowing you to refine policies without weakening your overall security posture.

Global (All Applications): Sources configured here define the minimum standard applied everywhere, ensuring unsafe patterns are blocked and trusted sources are consistently enforced.
Application: Sources defined here are applied to all hosts within the specific application.
Host: Sources defined here apply to a specific website domain. It’s at this level that the most specific configuration should be made. If you configure a specific Edit host, then you can have separate Content Security Policies for both the front-end and the CMS.
Page: Sources can be defined at this level by designing Content Pages and Experiences that implement IContentSecurityPolicyPage. This allows editors to include an iFrame on a specific page without opening the entire site for that domain.

So how would this work in practice? In my primary test instance I configured three separate applications, with each application having a Primary host and an Edit host. For my Global level, I implemented the ‘self’ source across most directives. For my Edit host I added all of the additional permissions required by the CMS backend, such as ‘unsafe-inline’ and https://*.optimizely.com. For my Primary host, I added required functionality such as nonce attributes. The end result was a much more consise and secure Content Security Policy tailored to specific journeys.

Permissions Policy and Response Headers

Permissions Policy and Response Headers follow a simpler model. These are configured at Global, Application, or Host level using an override approach where more specific levels replace broader ones. A configuration at Application level replaces the configuration at a Global level; a configuration at Host level replaces any configuration at the Application and Global configurations.

Cross Origin Resource Sharing

The Cross Origin Resource Sharing (CORS) configuration is applied at a webserver level by hooking into .NET Core built-in CORS functionality that addresses incoming traffic to the webserver. The implementation of this functionality does not yet support the Global, Application or Host level variations.

What does this mean for Optimizely CMS 12?

Support for CMS 12 is not going away. The new functionality introduced in version 7 will be backported into a version 6 release for CMS 12, where feasible. Ongoing development will continue to target CMS 13 first, with CMS 12 updates following based on demand and compatibility.

I can review the recent download stats on a version by version basis on nuget and I will use this as an indicator of the current market need. I expect that it will take a few years for a significant portion of customers to move from CMS 12 to CMS 13. Therefore, I expect to maintain the CMS 12 version for the short to mid term.

How does the change from Sites to Applications impact the user?

Optimizely CMS 13 introduces Applications in place of Sites, with a shift from GUID-based identifiers to string-based keys. This change breaks the direct relationship with existing configurations (such as security.txt) and requires structural updates to how data is stored.

To simplify migration, version 7 uses a new set of database tables rather than attempting to deal with conflicts between CMS 12 and CMS 13 versions. As a result, users will need to reconfigure headers and security.txt when upgrading, it is recommended that you use the built-in Import/Export functionality to carry settings across.

Summary

Stott Security is a free, open-source add-on for Optimizely CMS 12 and CMS 13, designed from the ground up to be accessible to both technical and non-technical users, with built-in audit functionality to provide clear accountability. The add-on supports both PaaS-based traditional headed and headless solutions.

Version 7 delivers a major update targeting Optimizely CMS 13 and .NET 10. Headers can now be defined at a Global, Application or Host level, providing a more flexible solution.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12 & 13. You can find all of my content collated on https://www.stott.pro/

Stott Robots Handler v7 for Optimizely CMS 13

Thu, 02 Apr 2026 09:44:40 GMT

Stott Robots Handler version 7 is now available for Optimizely PaaS CMS 13. This is includes updates to support the switch from a Site based architecture to an Application based architecture.

How does the change from Sites to Applications impact the user?

The data structure of Applications is subtly different to that of Sites. In CMS 12, sites were uniquely identified using a GUID. In CMS 13, Applications are uniquely identified using a string identifier. This breaks the relationship between existing robots.txt and llms.txt content and the sites they were configured to serve.

Users will have to reconfigure their robots.txt and llms.txt content when moving from CMS 12 to CMS 13. The content for both files is stored in the Dynamic Data Store and is matched to an active application before being displayed to the CMS Administrator or being served to end users.

The Environment Robots and the API Tokens are global configurations and are not bound to a specific site or application. When upgrading to version 7 and CMS 13, these features will retain their existing configuration.

What features does Stott Robots Handler have?

🤖 Manage robots.txt content by Application and specific Host URL
📄 Manage llms.txt content by Application and specific Host URL
🔒 Override robots meta tags and response headers at an environment level to prevent indexing lower environments in search engines.
🌐 APIs to support headless solutions
🌐 APIs to support modification of robots.txt and llms.txt content directly within Optimizely Opal

What does this mean for Optimizely CMS 12?

Support for version 6 of the Stott Robots Handler is not going away. New functionality will be developed directly for version 7 and then backported to version 6 where appropriate. I can review the recent download stats on a version by version basis on nuget and I will use this as an indicator of the current market need.

I expect that it will take a few years for a significant portion of customers to move from CMS 12 to CMS 13. Therefore, I expect to maintain the CMS 12 version for the short to mid term.

Getting Started

Stott Robots Handler v7 is available for Optimizely PaaS CMS 13 on nuget.org and on api.nuget.optimizely.com. You can see the full installation instructions and package information over on the github page, including how to configure the add-on for use with Opti Id.

Getting started can be as simple as:

// Install the nuget package
dotnet add package Stott.Optimizely.RobotsHandler

// Add this to your services
services.AddRobotsHandler();

// Add this to your configure method
services.UseRobotsHandler();

// Add this to your _ViewImports.cshtml if you are running traditional
@addTagHelper *, Stott.Optimizely.RobotsHandler

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12 & 13. You can find all of my content collated on https://www.stott.pro/

Announcing Stott Security Version 5.0

Thu, 05 Mar 2026 09:17:42 GMT

March 2026 marks the release of Stott Security v5, a significant update to the popular web security add-on for Optimizely CMS 12+, with more than 115,000 downloads across nuget.org and nuget.optimizely.com. Below is a high-level overview of what’s new in this release.

Custom Header Management
Audit Record Clean Up
Audit Record Search
Granular Settings Import

Before we get into the details, we all know that the release of Optimizely CMS 13 is imminent. A CMS 13 compatible version of Stott Security has already been developed and will be updated as we see more previews of CMS 13. It is my intention to have a day 1 release of this add-on.

Custom Headers

I have had multiple requests to add functionality that allows users to add or remove custom headers from the response. The data storage and UI for the existing Response Headers UI was inflexible, therefore I have rebuilt this feature from the ground up.

Users can add new headers with any valid header name structure and define a value and a behaviour. The three behaviours are:

Add: This will add the header to the response and will require a value to be specified.
Remove: This will remove the header from the response.
Disabled: No action will be performed for this header.

For traditional / in-process websites: the order of your middlewares and the point in time where the header is added will impact the success rate for removal of headers. The server header for example is not present in the Response object while middlewares are processed and as such will not be impacted. Also headers added after the response has been served will not be affected, this means headers added by CloudFlare for example will not be removed.

For Headless websites: the Header Listing API has been updated so that all configured headers now have an “isRemoval” property that highlights if the header should be removed or not.

Headless API Examples:

/stott.security.optimizely/api/compiled-headers/list/
/stott.security.optimizely/api/compiled-headers/list/?pageId=123

Example Response:

[
    {
        "key": "a-custom-header",
        "value": "a-value",
        "isRemoval": false
    },
    {
        "key": "Content-Security-Policy",
        "value": "default-src \u0027self\u0027;...", // Full CSP will be included
        "isRemoval": false
    },
    {
        "key": "server",
        "value": "",
        "isRemoval": true
    },
    {
        "key": "X-Content-Type-Options",
        "value": "nosniff",
        "isRemoval": false
    },
    {
        "key": "X-Xss-Protection",
        "value": "0",
        "isRemoval": false
    }
]

⚠️ Migration warning: Any configuration on the old Response Headers interface will need to be recreated. Response Headers that were previously managed through the old interface will appear as Disabled and the edit modal will present the same friendly options that were available in the previous UI. The headers that need re-configuring are limited to:

Cross-Origin-Embedder-Policy
Cross-Origin-Opener-Policy
Cross-Origin-Resource-Policy
X-Content-Type-Options
X-XSS-Protection
X-Frame-Options
Referrer-Policy
Strict-Transport-Security (HSTS)

Audit Record Clean Up

Stott Security has long had the ability to Audit changes to the configuration settings, but there has not been any means to clean up the audit records. A new scheduled job called [Stott Security] Audit Record Clean Up has been created that will remove audit records that exceed a configured retention period. By default this period is set to 2 years, but can be altered during the Add-On configuration by specifying the AuditRetentionPeriod.

services.AddStottSecurity(options =>
{
  options.ConnectionStringName = "EPiServerDB";
  options.NonceHashExclusionPaths.Add("/exclude-me");
  options.AuditRetentionPeriod = TimeSpan.FromDays(730); 
},
authorization =>
{
  authorization.AddPolicy(CspConstants.AuthorizationPolicy, policy =>
  {
    policy.RequireRole("WebAdmins");
  });
});

Audit Record Search

Filtering of audit records has always been present within the system, however it was limited to simple filters for user, operation, record and date range. Looking for very specific changes was a time consuming exercise. This change adds a text filter which will be used to find matches within the following fields:

Indicator
- Source for Content Security Policy Sources
- Directive for Permission Policy directives
- Header Name for Custom Headers
Old Value
New Value

🙏 Thank you to Samuel Joseph for delivering Add a free text filter to the Audit Screen.

Granular Settings Import

The ability to import and export settings was introduced back in version 2.6.0 but was straight forwards in terms of options. Over time this functionality was updated to support partial imports by ignoring root settings that were null. As part of this release, users now have the ability to choose specifically which settings they want to import.

🙏 Thank you to Samuel Joseph for providing the initial implementation of Enhance Settings Import Tool.

Summary

Stott Security is a free, open-source add-on for Optimizely CMS 12+, designed from the ground up to be accessible to both technical and non-technical users, with built-in audit functionality to provide clear accountability. The add-on supports both PaaS-based traditional headed and headless solutions.

Version 5 delivers a major update featuring a rebuilt Response Headers functionality with enhanced support for custom headers; enhanced Audit Management with configurable retention periods and new free-text audit search, and a more granular Settings Import tool that allows selective configuration.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Working With Applications in Optimizely CMS 13

Fri, 30 Jan 2026 12:10:23 GMT

💡Note: The following content has been written based on Optimizely CMS 13 Preview 2 and may not accurately reflect the final release version.

As part of my preparation of Stott Security and Stott Robots Handler for PaaS CMS 13, I’ve had to revisit key functionality in my add-ons that use site definitions. These add-ons allow a user to define robots.txt, llms.txt and security.txt configurations on a global, per site and per host definition basis. As a result I need to be able to read out data about the different site configurations.

Accessing Site / Application Definitions

With CMS 13 taking the lead from SaaS and thinking about more composable architectures, there is a shift from thinking about “sites” and instead thinking about “applications”. There is also a shift to having two different application configurations. There is the traditional “in-process” CMS application which is represented by the Website class, and there is the “headless” application which is represented by the RemoteWebsite class. These two applications both inherit the Application class.

The existing ISiteDefinitionRepository is now deprecated, but functions for now and only returns in-process applications. In order to access both application types you need to use the new interface of IApplicationRepository. This new repository exposes the following methods for listing applications:

// In-process applications
var data = await applicationRepository.ListAsync<Website>();

// Headless applications
var data = await applicationRepository.ListAsync<RemoteWebsite>();

// All applications
var data = await applicationRepository.ListAsync();
var data = await applicationRepository.ListAsync<Application>();

Another key change with here is that the GUID Id field is now defunct, and while ISiteDefinitionRepository still functions, the Id is populated with an empty GUID for compatibility purposes. The new objects use a name property instead and is generated as a sanitized lowercase version of the provided Display Name on creation and then becomes an immutable property.

Another welcome change with IApplicationRepository is the shift to asynchronous APIs, which makes it easier to write non-blocking and scalable code, particularly in high-throughput scenarios.

⚠️ Migration warning: If you previously keyed configuration by SiteDefinition.Id, you will need a migration strategy to map legacy GUIDs to Application.Name. There is no one-to-one replacement, and the new name is immutable once created.

Resolving The Current Application

Historically getting the current website for your current page would be accessed by SiteDefinition.Current. This is no longer available and instead we need to use IApplicationResolver instead. This provides access to the following methods:

// Retrieve the application based on a host name 
var result = applicationResolver.GetByHostname(hostName, fallbackToDefault);
var result = await applicationResolver.GetByHostnameAsync(hostName, fallbackToDefault, cancellationToken);

// Retrieve the application based on a given content reference
var app = applicationResolver.GetByContent(contentReference, fallbackToDefault);
var app = await applicationResolver.GetByContentAsync(contentReference, fallbackToDefault, cancellationToken);

// Retrieve the application based on the current HTTP Context
var app = GetByContext();
var app = await GetByContextAsync(cancellationToken);

Decompiling the DefaultApplicationResolver I can see that the GetByContext methods actually just wrap the GetByContent and GetByHostName methods and attempt to do this by Content first. If your functionality needs to operate outside of a content route then it could be better to retieve the application by host name directly, just bear in mind that this returns an ApplicationHostResolution rather than an Application.

public async ValueTask<Application?> GetByContextAsync(CancellationToken cancellationToken = default(CancellationToken))
{
    Application application = null;
    ContentReference routedContentLink = _routedContentLinkResolver.RoutedContentLink;
    if (!ContentReference.IsNullOrEmpty(routedContentLink))
    {
        application = await GetByContentAsync(routedContentLink, fallbackToDefault: true, cancellationToken).ConfigureAwait(continueOnCapturedContext: false);
    }

    if (application == null)
    {
        string hostName = _requestHostResolver.HostName;
        (application, _) = await GetByHostnameAsync(hostName, fallbackToDefault: true, cancellationToken).ConfigureAwait(continueOnCapturedContext: false);
    }

    return application;
}

Summary

CMS 13 replaces the traditional concept of sites with applications, supporting both in-process and headless models. While legacy site APIs still exist for compatibility, new development should use IApplicationRepository and IApplicationResolver.

The shift from GUID-based site IDs to immutable application names is the most significant change and has real implications for configuration storage and migration. Although this requires some adjustment, the new application model provides a cleaner and more flexible foundation for modern CMS architectures.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Announcing Stott Security Version 4.0

Fri, 16 Jan 2026 10:27:26 GMT

January 2026 marks the release of Stott Security v4, a significant update to the popular web security add-on for Optimizely CMS 12, with more than 109,000 downloads across nuget.org and nuget.optimizely.com. Below is a high-level overview of what’s new in this release.

Features
- Manage security.txt files
- Improved nonce and hash support
- UI rebuilt with Vite
- .NET 10 support
Bug fixes
- Conditional output of HSTS headers

Manage Security.txt Files

A security.txt file provides a standardized way for security researchers to report vulnerabilities by clearly defining contact details and disclosure policies. This helps ensure that security issues are reported responsibly and reach the appropriate team quickly. Stott Security allows you to manage security.txt content globally, per site, or even per host, and serves it from the standard path /.well-known/security.txt. You can read more about the specification on the official security.txt website.

The UI for managing security.txt content will be familiar to users of Stott Robots Handler. The core functionality has been carried across and enhanced with minor behavioural adjustments and security.txt specific guidance. While there are recommended formats for security.txt, real-world implementations vary widely, so this initial release supports free-form content entry and links to the official security.txt site, where a useful generator is available.

As with all features in Stott Security, changes to security.txt content are fully audited. However, security.txt files are intentionally excluded from the security header import and export functionality. Security headers are applied site-wide, whereas security.txt content can vary by site and host and is typically simple to copy between environments when required.

Improved Nonce and Hash Support

Previously, Nonce support for script and style elements was enabled via a single, all-or-nothing option in the CSP Settings screen. In version 4, the Generate Nonce and Use Strict Dynamic options have been removed from CSP Settings and are now configured as sources within the CSP Sources screen. This change allows for more granular control over CSP configuration. Existing installations will have their settings automatically migrated during upgrade to preserve current behaviour.

Support for hashes has also been added to the CSP Sources list, with validated support for SHA-256, SHA-384, and SHA-512. While hash-based CSP rules are now fully supported, they are generally not the recommended primary approach. Where possible, nonce attributes should be applied to script and style elements, inline scripts and styles should be avoided, and nonce support should be used within GTM-injected tags.

It is important to note that Optimizely CMS does not natively support nonce or hash attributes in the Editor and Admin interfaces. These interfaces do not apply nonce attributes to script or style elements, and the client-side Quick Navigator gadget injects page-specific data into inline script blocks. As a result, a unique hash would be required for every page. Enabling nonce or hash support therefore requires disabling the Quick Navigator widget.

To provide deterministic control over where nonce and hash functionality is applied, version 4 introduces configurable exclusion paths. Requests routed to these paths, or any sub-paths beneath them, will have nonce and hash sources automatically removed from the generated Content Security Policy.

services.AddStottSecurity(cspSetupOptions =>
{
    cspSetupOptions.ConnectionStringName = "EPiServerDB";
    cspSetupOptions.NonceHashExclusionPaths.Add("/exclude-me");
},
authorizationOptions =>
{
    authorizationOptions.AddPolicy(CspConstants.AuthorizationPolicy, policy =>
    {
        policy.RequireRole("WebAdmins", "Everyone");
    });
});

By default the following paths are already added to the Nonce and Hash exclusion list:

/episerver
/ui
/util
/stott.robotshandler
/stott.security.optimizely

Framework Updates

When I started Stott Security, the UI was built using Create React App and this was my first full React UI. Since then, the Create React App way of building React UI has been deprecated. To bring the UI up to date, I have rebuilt the UI using Vite and I’ve taken the opportunity to update all dependencies.

An extra compilation target has been added for .NET 10. Historically I’ve had to compile for multiple versions of .NET due to breaking changes in Entity Framework. In order to maintain backwards compatibility for existing users, this now means I am compiling for .NET 6, 8, 9 and 10. When CMS 13 is released, I will be reducing my compilation targets to just .NET 10 for CMS 13.

Small Changes

The Strict-Transport-Security header will now only be included in HTTPS responses in alignment with best practices. See: MDN : Strict-Transport-Security header
The “Add Content Delivery API Headers” button on the CORS has been renamed and will add all of the following headers for both the Content Delivery and Definition APIs:
- x-epi-contentguid
- x-epi-branch
- x-epi-siteid
- x-epi-startpageguid
- x-epi-remainingroute
- x-epi-contextmode
- x-epi-continuation

Closing Thoughts

Stott Security is a free, open-source add-on for Optimizely CMS 12, designed from the ground up to be accessible to both technical and non-technical users, with built-in audit functionality to provide clear accountability. The add-on supports both PaaS-based traditional headed sites and headless solutions.

For existing users, this release aims to enhance flexibility and modernise key areas without disrupting established configurations. For those new to Stott Security, version 4 provides a solid, standards-aligned foundation for securing Optimizely CMS applications.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Triggering Opal Workflows When Publishing Content

Mon, 05 Jan 2026 09:05:42 GMT

Over the course of November, my colleagues at Netcel and I took part in Optimizely’s Opal Innovation Challenge. We were tasked to be inventive and to discover new ways in which we could use Opal with emphasis on Specialized Agents, Workflows and Tools. If you are unaware of what these features are, my colleague Graham Carr has written a great introduction blog entitled A day in the life of an Optimizely OMVP - Optimizely Opal: Specialized Agents, Workflows, and Tools Explained.

In this technical blog, I’m going to focus on how we can leverage publish events to trigger workflows within Opal. The traditional method of doing this would be to leverage content events in C# by registering handlers for the IContentEvents interface. This approach has the downside of only being applicable for PaaS solutions. I will be showing you how to achieve the same goal using webhooks in Content Graph which has the advantage of being applicable to both SaaS and PaaS CMS without relying on CMS hosted event handlers.

💡Tip: Optimizely is moving away from Search & Navigation to Content Graph for improved flexibility, stability and performance. Use add-ons such as OptiGraphExtensions to manage your synonyms in Content Graph.

Solution Flow

Optimizely Content Graph has webhook functionality that will allow you to subscribe to indexing events. If you’re developing headless solutions with Content Graph you should already be familiar with this functionality. For this solution the data journey will be as follows:

User publishes a page or experience in the CMS
The content changes are indexed into Content Graph
Content Graph will send a notification to an Integration API using webhooks
The Integration API will retrieve additional information from Content Graph
The Integration API will package the data up and send it to a webhook endpoint in Optimizely Opal

You might ask why there is an Integration API included here? This is because Content Graph webhooks intentionally provide minimal payloads, which keeps them fast and reliable but requires enrichment downstream.

Testing Webhooks in Third Party Systems

Testing Webhook responses can be a challenge, especially if you are unsure what real data is going to look like. This is where tools such as WebhookCool come in. This particular website provides you with a temporary end point that you can point your webhook at so that you can examine your data before implementing an endpoint. I’ve found this particular tool to be very helpful during the prototyping phases.

🔒 Security Warning: Third party tools like WebhookCool can be useful for testing APIs, however you should never use them with a production environment or anything that might contain PII data.

Creating Webhooks In Content Graph

A webhook can be registered with Content Graph using a POST request. This POST request will require authorization using Basic or HMAC Authentication. In this example I am registering the webhook using Basic Authentication with the following key pieces of information:

The request contains the URL and method for the webhook.
The topic is set to be “doc.updated”, this tells Content Graph we are only interested in single document update events.
The filter is set to filter content to published items only.

POST https://cg.optimizely.com/api/webhooks
Content-Type: application/json
Authorization: Basic <Base64 encoded App Key and Secret>

{
    "request" : {
        "url" : "https://webhook.cool/at/randomly-generated-name",
        "method": "post"
    },
    "topic": ["doc.updated"], 
    "filters": [
        {
            "status": { "eq": "Published" }
        }
    ]
}

When you update the content item in Content Graph you will get one or more webhook events being executed with a response like so:

{
  "timestamp": "2025-12-22T13:56:00.7410789+00:00",
  "tenantId": "00000000000000000000000000000000",
  "type": {
    "subject": "doc",
    "action": "updated"
  },
  "data": {
    "docId": "662e2c97-c658-497d-a998-1c00df22e600_en_Published"
  }
}

Out of the box this isn’t ready for consumption for Opal as this doesn’t contain any information about the updated page beyond the docId which is a unique identifier within graph for the content item. If you wanted to pass something more useful to Opal, like a URL, then you will need an Integration API end point to handle this.

💡Tip: Please note that webhooks are registered per CMS and Content Graph environment, typically these are Integration, Preproduction and Production. However, there is only one Opal instance and you may not want events raised during QA or UAT to burn through your Opal credits.

Creating The Integration API

🔒 Please Note: identifiers in these examples have been swapped out with a randomly generated id to aid visualization and should not reflect any specific system.

The first thing we will need is to convert the Content Graph docId into data we can use to filter graph queries. This requires us to split the docId based on underscores with the first component being the Content GUID, the second being the language for the content and the third being the published state. To make matters more interesting, the Content GUIDs are stored without hyphens in Content Graph, so we have to strip those out too:

public bool TryConvertContentIdentifiers(string docId, out string id, out string locale)
{
    id = string.Empty;
    locale = string.Empty;
    if (string.IsNullOrWhiteSpace(docId))
    {
        return false;
    }

    var components = docId.Split('_');
    if (components.Length < 2 || !Guid.TryParse(components[0], out _) || string.IsNullOrWhiteSpace(components[1]))
    {
        return false;
    }

    id = components[0].Replace("-", string.Empty);
    locale = components[1];
    return true;
}

This will essentially turn 662e2c97-c658-497d-a998-1c00df22e600_en_Published into 662e2c97c658497da9981c00df22e600 and en. We can then use these to request URL data from Content Graph with our query looking like:

query MyQuery {
  _Page(
    orderBy: { _modified: DESC }
    where: {
      _metadata: { key: { eq: "662e2c97c658497da9981c00df22e600" }, locale: { eq: "en" } }
    }
  ) {
    items {
      _id
      _metadata {
        url {
          base
          hierarchical
          default
        }
      }
    }
  }
}

This will have a response that looks like:

{
  "data": {
    "_Page": {
      "items": [
        {
          "_id": "662e2c97-c658-497d-a998-1c00df22e600_en_Published",
          "_metadata": {
            "url": {
              "base": "https://www.example.com",
              "hierarchical": "/insights/",
              "default": "/insights/"
            }
          }
        }
      ]
    }
  },
  "extensions": { ... }
}

We can then wrap this up into a request to get the data from Content Graph by performing the following:

Using a StringBuilder to create the query.
Retrieving the data from Content Graph using single key authentication.
Serializing the data onto an object model based on the example response above.
Validating the URL data for each content item in the response

public async Task<List<Uri>> GetContentUris(string id, string locale)
{
    if (string.IsNullOrWhiteSpace(id) || string.IsNullOrWhiteSpace(locale))
    {
        return [];
    }

    // Build GraphQL Query to fetch content details
    var stringBuilder = new StringBuilder();
    stringBuilder.AppendLine("query MyQuery {");
    stringBuilder.AppendLine("  _Page(");
    stringBuilder.AppendLine("    orderBy: { _modified: DESC }");
    stringBuilder.AppendLine("    where: {");
    stringBuilder.AppendLine($"      _metadata: {{ key: {{ eq: \"{id}\" }}, locale: {{ eq: \"{locale}\" }} }}");
    stringBuilder.AppendLine("    }");
    stringBuilder.AppendLine("  ) {");
    stringBuilder.AppendLine("    items {");
    stringBuilder.AppendLine("      _id");
    stringBuilder.AppendLine("      _metadata {");
    stringBuilder.AppendLine("        types");
    stringBuilder.AppendLine("        key");
    stringBuilder.AppendLine("        locale");
    stringBuilder.AppendLine("        published");
    stringBuilder.AppendLine("        url {");
    stringBuilder.AppendLine("          base");
    stringBuilder.AppendLine("          hierarchical");
    stringBuilder.AppendLine("          default");
    stringBuilder.AppendLine("        }");
    stringBuilder.AppendLine("      }");
    stringBuilder.AppendLine("    }");
    stringBuilder.AppendLine("  }");
    stringBuilder.AppendLine("}");

    // Get the data from Content Graph using a GET request using single key authentication
    var query = stringBuilder.ToString();
    var authKey = configuration.GetValue<string>("GraphSingleKey");
    var graphUri = new Uri($"https://cg.optimizely.com/content/v2?auth={authKey}&cg-roles=Content%20Editors&query={Uri.EscapeDataString(query)}");

    var httpClient = httpClientFactory.CreateClient();
    var response = await httpClient.GetAsync(graphUri);

    response.EnsureSuccessStatusCode();

    var content = await response.Content.ReadAsStringAsync();
    var contentResponse = JsonSerializer.Deserialize<ContentQueryResponseDto>(content)!;

    // Validate the results so we only return URLs that are valid absolute URLs.
    var pages = contentResponse?.Data?.Pages?.Items ?? [];
    var contentUris = new List<Uri>();
    foreach (var page in pages)
    {
        var contentUrl = $"{page.Metadata?.Url?.Base}{page.Metadata?.Url?.Hierarchical}";
        if (Uri.IsWellFormedUriString(contentUrl, UriKind.Absolute))
        {
            contentUris.Add(new Uri(contentUrl));
        }
    }

    return contentUris;
}

Now we have a means to transform the data from the Graph Webhook and to retrieve the URL data from Content Graph, we can use this inside of a C# Controller to do the following:

Act as the endpoint for the Content Graph Webhook
Retrieve the URLs for the updated content items from Content Graph
Push each URL into the Opal Workflow Webhook

[HttpPost]
[Route("api/content/publish")]
[Consumes("text/plain", "application/json")]
public async Task<IActionResult> ContentPublishEvent([FromQuery] string apiKey)
{
    // [FromBody] attribute does not work with a content-type of text/plain.
    // So read the response body out manually and then deserialize it.
    WebhookEventDto? contentPublishEvent;
    using (var reader = new StreamReader(Request.Body, Encoding.UTF8))
    {
        var body = await reader.ReadToEndAsync();
        if (string.IsNullOrWhiteSpace(body))
        {
            return BadRequest();
        }

        contentPublishEvent = JsonSerializer.Deserialize<WebhookEventDto>(body);
    }

    // Validate the request payload has the required data items
    if (string.IsNullOrWhiteSpace(contentPublishEvent?.Data?.DocId) || string.IsNullOrWhiteSpace(apiKey))
    {
        return BadRequest();
    }

    // Convert Identifiers from the Content Graph Webhook data into values to locate the item in Content Graph
    if (!TryConvertContentIdentifiers(contentPublishEvent.Data.DocId, out var id, out var locale))
    {
        return BadRequest();
    }

    // Get Page Uris from Content Graph
    var pageUris = await GetContentUris(id, locale);

    // Send Page Uri's to the Opal Workflow Webhook
    foreach (var pageUri in pageUris)
    {
        await PostAsync(opalUri, new { url = pageUri.ToString() });
    }

    return Ok();
}

public async Task PostAsync<TRequest>(string url, TRequest requestBody)
{
    var httpClient = httpClientFactory.CreateClient();
    var jsonContent = new StringContent(JsonSerializer.Serialize(requestBody), Encoding.UTF8, "application/json");
    var response = await httpClient.PostAsync(url, jsonContent);
    
    response.EnsureSuccessStatusCode();
}

Keen observers may note that the controller is configured to accept both text/plain and application/json payloads. In practice, Content Graph webhooks can deliver requests with a text/plain MIME type, while tools such as HTTP files or Postman typically send payloads as application/json. Since standard C# model binding does not run for requests with a text/plain content type, the request body is read and deserialized manually within the controller.

The end result of this integration API is that content update actions for non-pages will be ignored and Opal will only recieve an instruction to run a workflow for publicly accessible pages with a provided URL in a simple payload:

{ 
  "url": "https://www.example.com/insights/"
}

Creating the Workflow in Opal

A workflow in Optimizely Opal is a collection of one or more specialized agents that can perform a set of actions either as a direct result of a chat prompt or independently from the chat prompt based on triggers. One of these triggers is a Webhook trigger. In order to create a new Opal Workflow you will need to do the following:

Login to Opal at https://opal.optimizely.com/
Click on Agents in the left-hand menu
Click on the “Add Agent” drop-down CTA
Click on “Workflow Agent”
Enter a name and unique id for the agent
Click on “Edit Workflow”

At this point you should have the workflow editor screen showing, this is a drag and drop interface that allows you to add triggers, logic gates and agents and to link them together.

In this example I have created a new workflow trigger and set the payload so that it knows to expect a simple JSON object containing a URL. I have also given the workflow a chat trigger and a chain of two specialized agents. When you save your workflow trigger for the first time, the Webhook URL is automatically generated using the following format:

https://webhook.opal.optimizely.com/webhooks/<instance-id>/<webhook-trigger-id>
https://webhook.opal.optimizely.com/webhooks/abcdef0123456789abcdef0123456789/abcdef01-2345-6789-abcd-ef0123456789

This Webhook URL is what will be referenced by our Integration API.

End to End

Now we have all the components needed to bring the solution flow together by performing the following:

Creating the Opal Workflow Webhook.
Deploying the Integration API and configuring environment variables:
- Content Graph Single Key: For public content graph consumption and retrieval of page meta data.
- Opal Workflow Url: So that the Integration API can send payloads to the Opal Workflow
Register the Content Graph Webhook: So that content update events are sent to the Integration API

This approach allows content publishing in Optimizely CMS to automatically trigger AI-driven workflows in Optimizely Opal, regardless of whether the CMS is hosted on SaaS or PaaS. Content Graph provides lightweight publish events, the Integration API enriches those events with a public URL, and Opal executes a workflow using a simple, stable webhook payload.

Within Opal, you can then configure the workflow to orchestrate a chain of specialized agents, each focused on a specific responsibility. Because agents receive only a URL, they remain decoupled from CMS internals and can evolve independently over time.

Typical agents in this workflow might include:

Automated GEO Recommendations – analysing content structure and semantics to improve visibility in AI-driven search and generative experiences.
Automated SEO Assessment – evaluating headings, metadata, internal linking, and page structure to surface actionable optimisation insights.
Automated Brand Assessment – checking tone, messaging, and consistency against brand guidelines at publish time.

By combining these agents into a single workflow, teams can receive immediate, automated feedback on published content, reducing manual review effort while keeping the integration scalable, flexible, and CMS-agnostic.

Other Considerations

When building integrations like this, it’s important to consider security and resilience from the outset. As part of this solution, you should consider the following best practices:

Protect the Integration API
- Register the Content Graph webhook with an API key passed via the query string and validate it in the Integration API.
- This helps prevent unauthorised requests if the endpoint URL is discovered.
Secure the Opal Workflow webhook
- Configure the authorization parameters on the Opal Workflow webhook trigger.
- Ensure the Integration API includes the required authorization details when invoking the workflow.
- This prevents third-party actors from triggering workflows if the webhook URL is exposed.
Harden the Integration API
- Implement structured logging to support monitoring and diagnostics.
- Add robust error handling and validation.
- Use retry mechanisms to handle transient failures when calling external services.
- Apply brute-force and rate-limiting protections to reduce abuse risk.

References

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Creating Opal Tools Using The C# SDK

Wed, 03 Dec 2025 09:16:55 GMT

Over the last few months, my colleagues at Netcel and I have partaken in two different challenge events organised by Optimizely and centered around Opal. Opal is Optimizely’s agentic marketing AI, deeply integrated across the Optimizely One platform. What transforms Opal from a conversational assistant into a true content-orchestration engine is its use of Agents, Workflows and Opal Tools. Opal tools are APIs that allow Opal to perform functional tasks that would otherwise fall outside the AI’s native capabilities. Throughout August, our teams were looking at building Opal Tools as part of the Optimizely Opal Hackathon. Throughout November we took part in the Opal Innovation Challenge which saw us leveraging new capabilities using Opal Tools, Specialized Agents, Instructions and Workflows.

Optimizely currently provides SDKs in three languages for building Opal Tools: C#, JavaScript, and Python. For C#, Optimizely offers two separate SDKs: OptimizelyOpal.OpalToolsSDK and Optimizely.Opal.Tools. At the time of writing, the documentation and training materials primarily reference OptimizelyOpal.OpalToolsSDK, which only reached version 0.1.0 in May 2025. In contrast, Optimizely.Opal.Tools appears to be receiving more active development, with version 0.4.0 being released in September 2025.

Both C# SDKs can deliver Opal tools, we used OptimizelyOpal.OpalToolsSDK as part of the Hackathon and Optimizely.Opal.Tools as part of the innovation challenge. There are some differences in terms of capabilities for both SDKs which are as follows:

Feature	OptimizelyOpal.OpalToolsSDK	Optimizely.Opal.Tools
Automatic Discovery API	✔	✔
Tools API	✔	✔
Supports Tool Authentication	✔	✔
Supports Dependency Injection	✘	✔
Supports Island Functionality	✘	✔

While there isn’t a lot between them and both are pre-version 1.x, Optimizely.Opal.Tools is the stronger and more flexible offering.

Getting Started

This guide assumes you are familiar with C# and creating basic web applications. In your IDE of choice, create a new C# Web API project and install the Optimizely.Opal.Tools nuget package

Creating a Tool

In this example, we’ll create a simple “Hello World” tool. Every Opal Tool must have a parameters class, which can contain zero or more properties. Start by creating a new class with a single property named FirstName. If you decorate the property with the [Required] attribute, Opal will know that this parameter must always be provided. You should also add a [Description] attribute to help Opal understand the purpose of the property. Clear descriptions improve Opal’s ability to supply correct values.

public class HelloWorldParameters
{
    [Required]
    [Description("The first name of the person to greet.")]
    public string FirstName { get; set; } = string.Empty;
}

Next, create a class that serves as the entry point for your tool. Add a public method that returns either an object or Task<object> if your code can run asynchronously. Decorate the method with the [OpalTool] attribute to declare the tool’s name (using snake_case or kebab-case). Decorate the method with the [Description] attribute to explain when and how Opal should use it. Whatever you return must be serializable to JSON.

public class HelloWorldTools(IHelloWorldService service)
{
    [OpalTool("say-hello-world")]
    [Description("Says hello based on a provided first name")]
    public object HelloWorld(HelloWorldParameters parameters)
    {
        return new
        {
            Message = service.SayHello(parameters.FirstName)
        };
    }
}

Here we’re using dependency injection to provide an IHelloWorldService. This is supported only in the Optimizely.Opal.Tools SDK. You could also return a concrete type, but an anonymous object keeps the example simple.

Now set up your tools in Program.cs or Startup.cs. The AddOpalToolService() extension registers the tool infrastructure and must be called before registering any tools. Then call AddOpalTool<ToolClass>() for each tool class. Finally, use MapOpalTools() to make the endpoints available in the request pipeline:

var builder = WebApplication.CreateBuilder(args);

// Register tool dependencies:
builder.Services.AddScoped<IHelloWorldService, HelloWorldService>();

// Register Opal Tool Service and Tools
builder.Services.AddOpalToolService();
builder.Services.AddOpalTool<HelloWorldTools>();

var app = builder.Build();

// Add the Opal tools into the HTTP request pipeline
app.MapOpalTools();

app.Run();

Testing Your Tools

Because Opal Tools are invoked by Opal itself, the only way to test them locally is through API testing tools such as Postman or HTTP files. HTTP files allow you to version-control your test requests and run them directly from your IDE. Requirements for using HTTP files are:

Visual Studio: install version 17.14.19 or later (earlier versions offer no or only partial support).
Visual Studio Code: install the REST Client extension by Huachao Mao.

Here is an example HTTP file for testing both the discovery endpoint and the tool endpoint. Your IDE will render a “Send Request” link above each request:

### Variables for all tools
@Tools_HostAddress = https://localhost:8000
@token = dev-token-value
###

### 1. Tool Discovery Endpoint
GET /discovery
Accept: application/json
###

### 2. Say Hello

POST /tools/say-hello-world
Accept: application/json
Content-Type: application/json
Authorization: Bearer 

{
  "parameters": {
    "FirstName": "Mark"
  }
}

###

Executing the discovery endpoint from the HTTP file should provide the following response:

{
  "functions": [
    {
      "name": "say-hello-world",
      "description": "Says hello based on a provided first name",
      "parameters": [
        {
          "name": "FirstName",
          "type": "string",
          "description": "The first name of the person to greet.",
          "required": false
        }
      ],
      "endpoint": "/tools/say-hello-world",
      "auth_requirements": [],
      "http_method": "POST"
    }
  ]
}

Executing the tool endpoint should produce the following response:

{
  "message": "Hello Mark"
}

You’ll notice from the HTTP file example that the tool endpoint uses an Authorization header while the discovery endpoint does not. This is because the discovery endpoint must allow anonymous access. Tool endpoints can optionally require a bearer token. Note that this token must be static and shared by all tools within the same application. This is because you can only provide a single static value when registering your tools application in Opal.

Because the SDK controls the mapping of tool endpoints, you cannot apply individual [Authorize] attributes to them. If you want to require bearer-token authentication then you’ll need to implement custom middleware. That middleware should intercept requests whose path begins with /tools/ and validate that a bearer token is present (and valid) before allowing the request to proceed:

public sealed class ToolAuthenticationMiddleware
{
    private readonly RequestDelegate _next;

    public ToolAuthenticationMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        // Check if the request is for a tool endpoint
        if (context.Request.Path.StartsWithSegments("/tools"))
        {
            // Validate that the Authorization header is present and valid...
            if (!context.Request.Headers.TryGetValue("Authorization", out var authHeader) || !IsValidToken(authHeader))
            {
                context.Response.StatusCode = StatusCodes.Status401Unauthorized;
                await context.Response.WriteAsync("Unauthorized");
                return;
            }
        }

        await _next(context);
    }

    private bool IsValidToken(string authHeader)
    {
        // Validate the bearer token value here...
    }
}

Adding Authentication to a Tool

The bearer token discussed earlier is used solely to secure communication between Opal and your tool’s endpoint. It does not represent the identity of the Opal user. There may be scenarios where your tool needs to operate in the context of the authenticated user who initiated the request. To support this, Optimizely Opal can pass user authentication data to Opal Tools.

At the time of writing, the only supported authentication provider is “OptiId”. If your discovery endpoint includes a tool that references an unsupported provider, Opal will fail to process the discovery response. During initial tool registration, Opal provides feedback indicating that the provider is unsupported. If the tool has already been registered and you are syncing updates, Opal will return a success response but silently skip the update; something that can be quite misleading.

Let’s create a new tool that requires user authentication. To do this, you must apply the [OpalAuthorization] attribute to the method, specifying the provider (“OptiId”), the scope and whether authentication is mandatory. The method signature should include both a custom parameters object (like the previous example) and an instance of OpalToolContext, which contains the user’s authentication data and additional request context.

[OpalTool("test-auth")]
[Description("A tool to test auth")]
[OpalAuthorization("OptiId", "cms", true)]
public object TestAuthorization(AuthenticationTestParameters parameters, OpalToolContext context)
{
    return new
    {
        Provider = context?.AuthorizationData?.Provider,
        Details = context?.AuthorizationData?.Credentials,
        EmailAddress = parameters?.EmailAddress
    };
}

public class AuthenticationTestParameters
{
    [Description("The email address of the current opal user making this request.")]
    [Required]
    public string EmailAddress { get; set; } = string.Empty;
}

When inspecting the discovery endpoint for this tool, you’ll notice that the auth_requirements property is an array. This is intentional as Optimizely intends for Opal to support multiple authentication schemes for a single tool. If you want a tool to accept more than one provider, simply apply multiple [OpalAuthorization] attributes to the method.

{
  "functions": [
    {
      "name": "test-auth",
      "description": "A tool to test auth",
      "parameters": [
        {
          "name": "EmailAddress",
          "type": "string",
          "description": "The email address of the current opal user making this request.",
          "required": true
        }
      ],
      "endpoint": "/tools/test-auth",
      "auth_requirements": [
        {
          "provider": "OptiId",
          "scope_bundle": "cms",
          "required": true
        }
      ],
      "http_method": "POST"
    }
  ]
}

When Opal executes this tool, it sends the request body (shown below) to your endpoint. Notice that we decorated AuthenticationTestParameters.EmailAddress with a [Description] attribute of “The email address of the current Opal user making this request.”. Opal understands this description and will populate parameters.EmailAddress automatically based on the current user without any intervention being needed. This really highlights the ability for Opal to infer, supply, and validate contextual user data without intervention from the user or the tool. It can also give you a means to feed back to the user asynchronously using email.

{
  "parameters": {
    "EmailAddress": "jo.bloggs@example.com"
  },
  "auth": {
    "provider": "OptiID",
    "credentials": {
      "token_type": "Bearer",
      "access_token": "<access-token-1446-chars>",
      "org_sso_id": "<id>",
      "cfg_client_id": "",
      "user_id": "<guid-no-hyphens>",
      "instance_id": "<guid-no-hyphens>",
      "customer_id": "<guid-no-hyphens>",
      "product_sku": "OPAL"
    }
  },
  "environment": {
    "execution_mode": "interactive"
  },
  "chat_metadata": {
    "thread_id": "<guid>"
  }
}

Summary

In this article, we explored how to build Opal Tools with the Optimizely.Opal.Tools SDK, define parameters, handle authentication, and secure tool endpoints. You should now have a clear understanding of how Opal discovers, registers, and executes tools, as well as how user and bearer authentication data flow into your tool methods. With these fundamentals, you’re ready to create secure, functional tools that extend Opal’s capabilities.

Key Points:

Choose Optimizely.Opal.Tools over OptimizelyOpal.OpalToolsSDK - The newer SDK is more mature and feature rich.
All Opal Tool methods require a parameters class - If your tool takes no parameters, you must define an empty parameters class.
Good descriptions are critical for Opal’s understanding - Use [Description] attributes effectively to get the best results.
Discovery endpoints must allow anonymous access, tool endpoints should be secured with a bearer token.
Use [OpalAuthorization] and OpalToolContext for Opal Tools which require an authorized user context.
Use HTTP files for local testing of your discovery and tool endpoints.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Creating Opal Tools for Stott Robots Handler

Sun, 28 Sep 2025 08:22:40 GMT

This summer, the Netcel Development team and I took part in Optimizely’s Opal Hackathon. The challenge from Optimizely was to extend Opal’s abilities by creating tools that would wrap actual business flows allowing Opal to focus on inputs and outputs in a conversational context. Our initial submission was to develop event management tooling that would integrate with Optimizely SAAS Content Management System, Optimizely Content Management Platform and Eventbrite.

Optimizely has created SDKs in C#, JavaScript and Python to accelerate the process of developing Opal Tools, we opted to use the C# SDK due to language familiarity. This SDK required us to create static classes and methods that performed the tool actions while the SDK itself managed the routing for the tools and provided the discovery endpoint in its entirety. After delivering several tools for the hackathon I reflected on the SDK and how it was achieving our goals. As the routing was managed by the SDK, it really limited our ability to create our own controllers or to consider other hosting options such as Azure Functions. As the owner and maintainer of two Optimizely Add-ons I started to think about how this could work with PaaS CMS Add-ons.

I concluded that if I wanted to add Opal tools to my Add-ons that I would have to consider not using the SDK at all for the following reasons:

To avoid conflict with CMS Implementations that have tools as part of their delivery.
To avoid conflict with other CMS Add-ons that also attempted to use the SDK.
To keep Discovery and Tool endpoints in the same routing structure as the rest of my Add-on.
To apply custom bearer token validation and controller responses.

Opal Tool Requirements

When we consider that Opal Tools are simply REST APIs with a specific JSON requirement, we realise that this is something we have been delivering for years. There are essentially two flavours of endpoint; the discovery endpoint which describes your tools to Opal and then your tool endpoints themselves.

Discovery Endpoint

The discovery endpoint must return a JSON object containing an array of functions. Each function in this array describes a tool as below. Each entry must specify a name, description, endpoint, an array of parameters and a desired HTTP method. The description is especially important here as it will help Opal understand the scope and intent of your tool, getting this right is essential.

name: This is the name of your tool and should be all one word and unique.
description: This description will help Opal understand the intent for your tool, it’s important that this is meaningful.
parameters: This is a list of parameters Opal should send to your endpoint.
- name: This is the name of the parameter, this should match the case you expect to receive the data on.
- description: This description will help Opal understand how and what it should pass into this parameter.
- type: This tells Opal what type of data it should send you.
- required: This tells Opal if it should or could provide this parameter.
endpoint: This should be the endpoint for your API and must be relative to the discovery endpoint.
http_method: This tells Opal what HTTP method to use.
auth_requirements: This is an optional field and can be entirely omitted. Only populate this if you need an authentication to run your tool either from Opal or another identity provider.
- provider: The name of the identity provider.
- scope_bundle: The permission scope requested.
- required: An indicator as to whether this authentication method is required or not.

GET: /discovery

{
    "functions": [
        {
            "name": "myuniquetoolname",
            "description": "This is a description of what this tool will do.",
            "parameters": [
                {
                    "name": "parameterOne",
                    "type": "string",
                    "description": "A description of what data should be passed into this parameter.",
                    "required": false
                }
            ],
            "endpoint": "/tools/tool-api-endpoint",
            "http_method": "POST",
            "auth_requirements": [
                {
                    "provider": "OptiID",
                    "scope_bundle": "tasks",
                    "required": true
                }
            ]
        }
    ]
}

💡 Tip: Opal will not include the Authorization header when making a request to the discovery endpoint, so make sure your implementation is accessible anonymously.

Tool Endpoint

The endpoint for a tool must be relative to the discovery endpoint, i.e. it must exist beneath the discovery endpoint. If we assume that the discovery endpoint responds on https://www.example.com/path-one/discovery, and a tool has an endpoint of /tools/tool-one then Opal will send the request to https://www.example.com/path-one/tools/tool-one. It should be noted that if you register the with a trailing slash like https://www.example.com/path-one/discovery/, then Opal will send the request to https://www.example.com/path-one/discovery/tools/tool-one instead. Because of this, you may want to check on any redirect rules within your solution that force trailing slashes etc.

parameters: This will be a JSON object that has properties matching the defined tool parameters.
- parameterOne: This is just an example parameter; your own parameters defined in the discovery endpoint will appear here.
auth: This is an optional object and will only be provided if the tool has been specified as requiring a specific authentication.
- provider: This will be the name of the authentication provider.
- credentials: These are the specific authentication details.

Additional data items are also included as environment and chat_metadata, but these are not essential for the operation of your tool and could be useful in tracing operations. In the following example, a tool has been declared as having OptiId as an authentication requirement.

POST: /tools/tool-name

{
	"parameters": {
		"parameterOne": "Some value"
	},
	"auth": {
		"provider": "OptiID",
		"credentials": {
			"token_type": "Bearer",
			"access_token": "...",
			"org_sso_id": null,
			"user_id": "...",
			"instance_id": "...",
			"customer_id": "...",
			"product_sku": "OPAL"
		}
	},
	"environment": {
		"execution_mode": "interactive"
	},
	"chat_metadata": {
		"thread_id": "e597710c-2d10-4f07-9817-6fad9f2b748d"
	}
}

Implementing Opal Tools

Implementing the discovery endpoint itself is a simple. The C# SDK for example uses reflection and understands the attributes you decorate on your tools. As I’m not using an SDK in this scenario I could either ship a JSON object with my code, or in this case create the classes that achieve the same outcome. I opted to create the DTO objects that would be serialized into the desired JSON structure. I decorated the controller action with the HttpGet and AllowAnonymous attributes to ensure that the endpoint was publicly accessible for GET requests only.

[HttpGet]
[AllowAnonymous]
[Route("/stott.robotshandler/opal/discovery/")]
public IActionResult Discovery()
{
    var model = new FunctionsRoot { Functions = new List<Function>() };

    model.Functions.Add(new Function
    {
        Name = "getrobottxtconfigurations",
        Description = "Get a collection of robot.txt configurations optionally filtered by host name.",
        Parameters = new List<FunctionParameter>
        {
            new FunctionParameter
            {
                Name = "hostName",
                Type = "string",
                Description = "The host name to filter the robot.txt configurations by.",
                Required = false
            }
        },
        Endpoint = "/tools/get-robot-txt-configurations/",
        HttpMethod = "POST"
    });

    // More tools defined here...

    return CreateSafeJsonResult(model);
}

As I was aiming to support multiple endpoints with the same model structure, I used generics to create a wrapping object using the specific content model required by my endpoint. This is then added as a parameter to my controller actions with the FromBody attribute to ensure the model was pulled from the request body. Note that it is worth specifying the JsonPropertyName attributes as you cannot guarantee the serialization options of the hosting solution.

public class ToolRequest<TModel> where TModel : class
{
    [JsonPropertyName("parameters")]
    public TModel Parameters { get; set; }
}

public class GetRobotTextConfigurationsQuery
{
    [JsonPropertyName("hostName")]
    public string HostName { get; set; }
}

If your tool needs to have authentication passed into it, then you can extend this wrapping object to contain the authentication data like so:

public class AuthenticatedToolRequest<TModel> where TModel : class
{
    [JsonPropertyName("parameters")]
    public TModel Parameters { get; set; }

    [JsonPropertyName("auth")]
    public AuthData Auth { get; set; }
}

public class AuthData
{
    [JsonPropertyName("provider")]
    public string Provider { get; set; } = string.Empty;

    [JsonPropertyName("credentials")]
    public Dictionary<string, object> Credentials { get; set; } = new Dictionary<string, object>();
}

In the following example, I have decorated my controller with the HttpPost attribute and then two separate Route attributes. This helps my controller respond to both possible request paths, depending on whether the discovery endpoint was registered with a trailing slash or not. I have then used the generic ToolRequest<T> class to wrap my specific model of GetRobotTextConfigurationsQuery as the parameter for the method.

[HttpPost]
[Route("/stott.robotshandler/opal/tools/get-robot-txt-configurations/")]
[Route("/stott.robotshandler/opal/discovery/tools/get-robot-txt-configurations/")]
[OpalAuthorization(OpalScopeType.Robots, OpalAuthorizationLevel.Read)]
public IActionResult GetRobotTxtConfigurations([FromBody] ToolRequest<GetConfigurationsQuery> model)
{
    try
    {
        var configurations = _service.GetAll();
        if (!string.IsNullOrWhiteSpace(model?.Parameters?.HostName))
        {
            var hostName = model.Parameters.HostName.Trim();
            var specificConfiguration =
                configurations.FirstOrDefault(x => string.Equals(x.SpecificHost, hostName, StringComparison.OrdinalIgnoreCase)) ??
                configurations.FirstOrDefault(x => x.AvailableHosts.Any(h => string.Equals(h.HostName, hostName, StringComparison.OrdinalIgnoreCase)));

            if (specificConfiguration is null)
            {
                return Json(new
                {
                    Success = false,
                    Message = $"Could not locate a robots.txt config that matched the host name of {model.Parameters.HostName}."
                });
            }

            return Json(ConvertToModel(specificConfiguration, hostName, x => x.RobotsContent));
        }

        return Json(ConvertToModels(configurations, x => x.RobotsContent));
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "An error was encountered while processing the getrobottxtconfigurations tool.");
        throw;
    }
}

You may have noticed that my controller action also has an OpalAuthorization attribute. When you register your tools in Opal with the discovery endpoint, you have the option to provide a bearer token that will be sent in an Authorization header. This is a custom attribute that checks for the presence of an Authorization header with a bearer token and checks it against a user defined bearer token within my Add-on. In my Add-on I allow the user to define multiple bearer tokens with different read/write permissions like so:

Wrapping Up

At the end of the day, Opal tools are just REST APIs with a specific JSON contract. Before you start building, it’s worth validating which official SDK (JavaScript, Python, or C#) best suits your project. They provide a quick path to getting up and running. If you find the SDK constraints don’t fit your use case, building your own endpoints without an SDK gives you full control over routing, authentication, and integration with your existing codebase. This makes it easier to plug tools into your Optimizely CMS or Add-ons without worrying about conflicts and to consider serverless hosting such as Azure Functions if you are deploying your tools separately.

If you’re a developer working with Optimizely, I’d encourage you to give this a try yourself. Start small: build a discovery endpoint, define a tool with one or two parameters, and watch Opal call it directly. Once you see it working end-to-end, you’ll have a solid foundation for building more advanced tools tailored to your projects.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Raising and Retrieving Custom Events in Application Insights

Thu, 25 Sep 2025 08:49:35 GMT

Following on from Minesh's insight on how to extend Extending Application Insights in an Optimizely PaaS CMS Solution, I'd like to share another way you can use Application Insights to improve your offering. Application Insights has the ability to record and report on Custom Events that you raise within your application and raising an event is easy. I'm using this mechanism to track search terms without using a database and the examples below are about raising custom events during search and then retrieving the data back out and aggregating it.

Before you can get started with recording your own custom events, you need to prepare your application:

Install the Microsoft.ApplicationInsights.AspNetCore package
Retrieve and add the following settings from your Application Insights instance and add them to your appsettings.json:
- ApplicationInsights__ConnectionString
- ApplicationInsights__apiKey
- ApplicationInsights__AppId

Raising an Event

Raising an event is performed through the TelemetryClient which is part of the Microsoft.ApplicationInsights.AspNetCore package. The TrackEvent method takes two parameters, the name of your event and a dictionary<string, string> of custom data. The TelemetryClient will automatically add data such as the HTTP Method and Request Path along with other interesting information so you can focus on the details that matter to you. It goes without saying that you should not include PII data in your event data.

using Microsoft.ApplicationInsights;
using Microsoft.AspNetCore.Mvc;

public sealed class SearchController(ISearchService searchService, TelemetryClient telemetryClient) : Controller
{
    public async Task<IActionResult> Search(string? query)
    {
        LogTrackSearch(query);

        // Search Logic Goes Here
        var results = searchService.Search(query);

        return Json(results);
    }

    private void LogTrackSearch(string? query)
    {
        if (query is { Length: >3 })
        {
            telemetryClient.TrackEvent("TrackSearch", new Dictionary<string, string> { { "Query", query } });
        }
    }
}

You can then search for you custom events directly in the Logs section of Application Insights:

Retrieving Events

Now you have your events logged in Application Insights, you may want to retrieve this data back out to present it to an administrator. I strongly recommend you only use this functionality on an administration screen and do not use it to power the frontend of your website.

The first thing you will want to do is design your query, I recommend you do that directly in the Logs tool in Application Inisights. For my query, I only wanted data relating to my specific event type named "TrackSearch" and I wanted to aggregate that data by unique Query value and include a count of instances as UniqueCount. As a result my query looks like this:

customEvents
| where name contains "TrackSearch"
| where timestamp > ago(7d)
| where tostring(customDimensions["Query"]) != ""
| summarize UniqueCount=count() by Query=tolower(tostring(customDimensions["Query"]))
| project Query, UniqueCount
| order by UniqueCount desc

In order to retrieve this directly in your application, you need to make a GET request to Application Insights and include an escaped version of your query. You will recieve a response which contains a collection of tables. As my query is summarizing the customEvents table, that is the only populated table I receive in my response. Now the data isn't as straight forward as you might think. I had to include the following DTOs so I could serialize out the content:

public sealed class InsightsResponse
{
    [JsonPropertyName("tables")]
    public List<InsightsTable>? Tables { get; set; }
}

public sealed class InsightsTable
{
    [JsonPropertyName("name")]
    public string? Name { get; set; }

    [JsonPropertyName("columns")]
    public List<InsightsColumn>? Columns { get; set; }

    [JsonPropertyName("rows")]
    public List<List<object>>? Rows { get; set; }
}

public sealed class InsightsColumn
{
    [JsonPropertyName("name")]
    public string? Name { get; set; }

    [JsonPropertyName("type")]
    public string? Type { get; set; }
}

Then in my controller I use a StringBuilder to recreate my query, I then make a GET request to application insights and serialize out the response. The data I then have is not very usable in it's current form, so I have additional logic which then transforms the response data into a collection of TrackSearchTerm objects which is another DTO to contain the flat data I want to present back to the user.

public sealed class InsightsController(IConfiguration configuration) : Controller
{
    public async Task<IActionResult> SearchTerms(int numberOfDays = 7)
    {
        var stringBuilder = new StringBuilder();
        stringBuilder.AppendLine("customEvents");
        stringBuilder.AppendLine("| where name contains \"TrackSearch\"");
        stringBuilder.AppendLine($"| where timestamp > ago({numberOfDays}d)");
        stringBuilder.AppendLine("| where tostring(customDimensions[\"Query\"]) != \"\"");
        stringBuilder.AppendLine("| summarize UniqueCount=count() by Query=tolower(tostring(customDimensions[\"Query\"]))");
        stringBuilder.AppendLine("| project Query, UniqueCount");
        stringBuilder.AppendLine("| order by UniqueCount desc");
        
        var query = stringBuilder.ToString();
        var settings = GetApplicationInsightSettings();
        
        // Build the request URL
        string url = $"https://api.applicationinsights.io/v1/apps/{settings.AppId}/query?query={Uri.EscapeDataString(query)}";

        using var client = new HttpClient();

        // Set up API key in the header
        client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        client.DefaultRequestHeaders.Add("x-api-key", settings.ApiKey);

        // Send the request
        var response = await client.GetAsync(url);
        if (response.IsSuccessStatusCode)
        {
            var content = await response.Content.ReadAsStringAsync();

            var data = JsonSerializer.Deserialize<InsightsResponse>(content);
            var searchTerms = GetSearchTerms(data).ToList();

            return Json(searchTerms);
        }

        return Json(Enumerable.Empty<TrackSearchTerm>());
    }

    private static IEnumerable<TrackSearchTerm> GetSearchTerms(InsightsResponse? insightsResponse)
    {
        if (insightsResponse is not { Tables.Count: >0 })
        {
            yield break;
        }
        
        foreach (var table in insightsResponse.Tables)
        {
            var queryIndex = -1;
            var uniqueCountIndex = -1;

            if (table is not { Columns.Count: >0, Rows.Count: >0 })
            {
                continue;
            }

            // Get the numerical index of your column
            foreach (var column in table.Columns)
            {
                queryIndex = string.Equals(column.Name, "Query", StringComparison.OrdinalIgnoreCase) ? table.Columns.IndexOf(column) : queryIndex;
                uniqueCountIndex = string.Equals(column.Name, "UniqueCount", StringComparison.OrdinalIgnoreCase) ? table.Columns.IndexOf(column) : uniqueCountIndex;
            }

            foreach (var row in table.Rows)
            {
                yield return new TrackSearchTerm
                {
                    Query = queryIndex >= 0 ? row[queryIndex]?.ToString() : string.Empty,
                    UniqueCount = uniqueCountIndex >= 0 && int.TryParse(row[uniqueCountIndex]?.ToString(), out var uniqueCount) ? uniqueCount : 1
                };
            }
        }
    }
}

You can then present this result directly to your Administor in whatever shape you want. I do recommend adding additional logic to deduplicate search terms as you may end up with multiple events being executed due to delays in user input. For example you may get a responses of "Hello W" and "Hello World" from a single user if a search request is executed during a delay in user input. I haven't included that code here, but my complete solution includes it.

You now have all the information you need to start using custom events in application insights for your own needs. Happy Coding 😊

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Helping Opal Build LLMS.TXT Content

Fri, 19 Sep 2025 07:34:48 GMT

In my previous blog, Opal Tools Support Comes to Stott Robots Handler v6, I shared the news about new Opal Tools being built directly into the Stott Robots Handler AddOn for Optimizely PAAS CMS 12. In practice Opal handles generating and modifying robots.txt content effectively using these tools. However, it currently lacks the ability to structure or generate an valid llms.txt file. This is where instructions can help you achieve success.

What Are Instructions?

Opal instructions are written in markdown and help guide Opal in it's tasks. I asked Opal to describe Instructions and what their intent was and Opal had the following to say:

Opal Instructions are the guidelines and rules that govern my behavior and responses as the Optimizely AI Assistant. Their primary intent is to ensure that I:

1. Provide specific and actionable guidance: Instead of generic advice, I aim to give you concrete steps and relevant information.
2. Understand and utilize context: I use your current workflow, user information, and the Optimizely product you're using to tailor my responses.
3. Prioritize and use tools effectively: The instructions help me decide which tools to use (like creating campaigns, tasks, or searching documentation) and in what order, to best fulfill your request.
4. Focus on business outcomes: My responses are geared towards helping you achieve measurable results in digital optimization, experimentation, and customer experience.
5. Maintain a consistent and helpful persona: They guide my communication style, ensuring I use appropriate terminology and offer clear, concise, and thorough explanations.

In essence, these instructions are my operational manual, designed to make me a more effective and helpful assistant for you within the Optimizely platform.

Where to Manage Instructions

To view instructions, open the Optimizely Opal Interface and click on the instructions button in the left hand menu (book icon). Opal comes with a number of built-in instructions that you can view and use as examples for building your own instructions.

When you click on the Add Instruction Button, you will be provided with this screen, which will allow you to:

Name your Instruction
Enter the details for your instruction
Define where the instructions can be used.

You can read more about managing instructions over in Optimizely's support documentation here: Instructions overview

Creating Instructions for Generating LLMS.TXT Content

The instructions content should start with an overview, this should give Opal the context of what it will be doing and what it will be expected to do. Then define the execution steps; Each step should be built be built up of one or more bullet points that allow it to complete that step.

A bullet labeled **Prompt** tells Opal to request specific input from the user. If the user has already provided this in their own prompt, then Opal will recognize this and move onto the next bullet. A bullet labeled with **Action** will provide Opal with the knowledge of the actions it will need to perform before feeding back to the user.

In the following instructions, I have split the execution into three main steps:

Analysis
1. Request a URL for the home page of the website we are generating the llms.txt content for.
2. Review the requested page and generate a title, a synopsis and a collection of links ordered by section.
Generation
1. Use the information gained in the Analysis step to generate the llms.txt content using the example detailed in the Generation step as an example
Review
1. Show the generated llms.txt content to the user.
2. Ask them if they would like to save the llms.txt content and which host name they want to save it for
3. Save the llms.txt content using the tool provided by the Stott Robots Handler

Here is my example instructions content:

# Overview

When a user asks you to create an llms.txt file for a website.  You will enter an analysis mode for a provided website.  You will review the root page of the website and understand the business title for the website and generate a synopsis and collate a set of links found on that page.

Understanding this information you will then follow the execution steps below.

## Execution Steps

### Step 1 : Analysis
- **Prompt**: Request a home page URL for analysis
- **Action**: Review the content on the page and understand the following information:
  - A title for the website
  - A synopsis for the purpose of the website.
  - A collection of links ordered by section

### Step 2 : Generation

- **Action**: Using the information collated in Step 1. Create markdown content as per the following example structure:

```
# Title

> Optional description goes here

Optional details go here

## Section name

- [Link title](https://link_url): Optional link details
- [Link title](https://link_url): Optional link details
- [Link title](https://link_url): Optional link details

## Optional

- [Link title](https://link_url) : Optional link details
- [Link title](https://link_url): Optional link details
- [Link title](https://link_url): Optional link details
```

### Step 3 : Review

- **Action**: Show the generated llms.txt content to the user
- **Prompt**: Ask the user whether they will like to save the llms.txt content and which hostName they want to save it for
- **Action**: Save the content using the savellmstxtconfigurations tool

Before I created these instructions, when I asked Opal to generate an llms.txt file, it would typically provide me with a content not too dissimilar to robots.txt style content. Using these instructions I was able to get Opal to scan my personal website and generate a starting llms.txt.

Note that I have trimmed down this generated output for brevity:

# Title: Mark Stott, Solutions Architect, Optimizely MVP

> This website is the personal blog and portfolio of Mark Stott, a Software Solution Architect and Optimizely MVP. It highlights his professional experience, specializations, and recent technical articles.

## Mark Stott: Software Solution Architect & OMVP

- [GitHub](https://github.com/GeekInTheNorth/)
- [LinkedIn](https://www.linkedin.com/in/mark-antony-stott/)

## Latest Posts

- [Opal Tools Support Comes to Stott Robots Handler v6](https://www.stott.pro/article/stott-robots-handler-v6): Published: 17th September 2025, Description: Stott Robots Handler for Optimizely CMS 12 has been released with added support for Opal Tools. Categories: Development, Optimizely, Stott Robots Handler
- [Simple Feature Flags In Optimizely CMS](https://www.stott.pro/article/simple-feature-flags-in-optimizely-cms): Published: 31st August 2025, Description: A simple approach to feature flagging within Optimizely CMS. Categories: Development, Optimizely
- [LLMS.txt support comes to Stott Robots Handler v5](https://www.stott.pro/article/stott-robots-handler-v5): Published: 8th August 2025, Description: Stott Robots Handler for Optimizely CMS 12 has been released with added support for llms.txt files. Categories: Development, Optimizely, Stott Robots Handler
- [Optimizing Content Security Policies to Stay Within HTTP Header Limits](https://www.stott.pro/article/optimize-your-csp): Published: 1st August 2025, Description: Discover how to audit, reduce, and optimize Content Security Policies. Helping you stay within browser and CDN header size limits. Categories: Development, Optimizely, Stott Security
- [Stott Security Version 3](https://www.stott.pro/article/stott-security-3): Published: 8th April 2025, Description: A summary of all new functionality and changes that have been introduced to the Stott Security module so far. Categories: Development, Optimizely, Stott Security

## Article Categories

- [Optimizely](https://www.stott.pro/article/list/optimizely)
- [Development](https://www.stott.pro/article/list/development)

In Summary

Opal Tools can be used to grant new functionality to Opal, but it is through the power of instructions that you can go from functionality to high impact. With more investment into the llms.txt instructions, you can help Opal craft great content for your llms.txt that is more specific to your business and the websites you release your content on. Opal Tools and llms.txt functionality is available in Stott Robots Handler v6 for Optimizely PAAS CMS 12 now.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Opal Tools Support Comes to Stott Robots Handler v6

Wed, 17 Sep 2025 09:03:03 GMT

During August, the Netcel team and I took part in the Optimizely Opal Hackathon. The challenge for this hackathon was to generate tools that extend Opal's abilities within the Optimizely Platform. As a late second entry for our team, I put forward a proposal to add Opal Tools support to an Optimizely AddOn. This work sparked something bigger, and today I'm excited to share that Opal Tools functionality and improved headless support has made it's way into a production-ready release of Stott Robots Handler v6.

This update makes it easy to manage your robots.txt and llms.txt content through Opal with secure token management.

Managing Bearer Tokens

Optimizely’s Opal uses bearer tokens for managing secure access to the tools. In version 6, A new tab has been added to the robots interface that allows the user to create and manage their bearer tokens. This screen not only provides supporting details around the available endpoints, but also lets you create multiple tokens, each with it’s own permissions for the robots.txt and llms.txt content.

Creating a token only takes a few clicks:

Click Add Token.
Give your token a name.
Generate it’s value.
Choose permission scopes for robots.txt and llms.txt.

It is important to remember that token values are only shown once. Do remember to take a copy of your token value before saving, because once it dissappears, you won’t see it again.

The available scopes are:

None: This grants no access to any endpoint.
Read: This grants access to the Get endpoint only.
Write: this grants access to both the Get and Save endpoints.

Registering The Tools in Opal

Adding the tools into Opal is a straightforward registration process.

Open the Optimizely Opal interface.
Click Tools (bottom of the left-hand menu).
From the tools screen, click Add Tool Registry.

You will see a short form:

Registry Name: Any value works; I recommend “Robots Handler Tools”
Discovery URL: On the Robots Handler token screen, use the Copy URL button next to the Discovery API and paste it here.
Bearer Token: Paste the token you generated earlier.

Using The Tools

Once registered, you can start using the Robots Handler tools immediately in Opal’s chat interface. Just ask questions or give instructions, such as:

“Get me a list of robots.txt configurations.”
“Show me the robots.txt for mydomain.com.”
“Add a sitemap.xml to the robots.txt for mydomain.com.”

It is best practice to be as verbose as possible when getting Opal to make changes to your robots.txt or llms.txt content. Ask Opal to show you the changes first, review them and iterate before applying your changes.

One thing to note: the discovery endpoint itself doesn’t send a token. This means Opal sees all tools as potentially available, but actual access depends on the permissions you granted. If a tool requires more permissions than your token allows, Opal will receive a “not authorised” response.

Getting Started

Stott Robots Handler v6 is available for Optimizely PAAS CMS 12 on both nuget.optimizely.com and on nuget.org. You can see the full installation instructions and package information over on the github page, including how to configure the addon for use with Opti Id.

Getting started can be as simple as:

// Install the nuget package
dotnet add package Stott.Optimizely.RobotsHandler

// Add this to your services
services.AddRobotsHandler();

// Add this to your configure method
services.UseRobotsHandler();

// Add this to your _ViewImports.cshtml if you are running traditional
@addTagHelper *, Stott.Optimizely.RobotsHandler

With Opal Tools support now built into Robots Handler, managing your search visibility and AI indexing just got a lot more powerful and easier. Try it out today, and let me know how you use it in your projects.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

Simple Feature Flags In Optimizely CMS

Sun, 31 Aug 2025 17:56:00 GMT

The Problem

I was working with a CMS 11 client who wanted to introduce two new third party content sources. These would be synchronized into new specific page types that they would then surface alongside content written directly within the CMS on their listing pages. Their listing page has the ability to select a subset of content types which will be allowed within the results of the current listng page. This functionality allows them to use the same Listing Page type to act as a Case Studies listing page or a News Article listing page.

Different parts of the business would be sponsoring each new integration and there was an overlap in terms of related content types and functionality they would be referenced on. Rather than getting into regular merge conflicts with different branches implementing changes on the same objects I decided to adopt a feature flagging approach.

The Solution

The first requirement was a static object with a method that could be accessed globally with minimal overhead. As this was a CMS 11 solution built on .NET Framework, I placed the flags in the AppSettings section of the web.config file and accessed them through the ConfigurationManager.

public static class FeatureFlagProvider
{
    public const string FeatureOne = "FeatureOne:IsEnabled";
    public const string FeatureTwo = "FeatureTwo:IsEnabled";

    public static bool IsFeatureFlagActive(string featureFlag)
    {
        var featureFlagValue = ConfigurationManager.AppSettings[featureFlag];
            
        if (bool.TryParse(featureFlagValue, out var result))
        {
            return result;
        }

        return false;
    }
}

The solution includes a Site Settings content type that holds global content and settings for the site. Both integrations introduced options on the Site Settings content type and added properties to the Listing Page content type. These properties needed to be conditionally hidden based on feature flags. To achieve this, I created a custom attribute inheriting from ScaffoldColumn:

[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field, AllowMultiple = false)]
public class FeatureFlaggedColumnAttribute : ScaffoldColumnAttribute
{
    public FeatureFlaggedColumnAttribute(string featureFlag) : base(FeatureFlagProvider.IsFeatureFlagActive(featureFlag))
    {
    }
}

The ScaffoldColumn attribute accepts a single boolean indicating whether a property should be visible in the editor UI. My FeatureFlaggedColumn attribute instead accepts a string and converts it into a boolean using the FeatureFlagProvider which is passed into the base constructor. Applying this to a property on a content type looks like this:

[FeatureFlaggedColumn(FeatureFlagProvider.FeatureOne)]
public virtual ContentReference FeatureOneProperty { get; set; }

The Listing Page also had a multi-select field allowing editors to select one or more content types to filter Search & Navigation results. I needed to hide the new page types in the selection factory for this field without removing the field itself. The same FeatureFlagProviderwas used inside the selection factory:

public class ContentTypesSelectionFactory : ISelectionFactory
{
    public IEnumerable<ISelectItem> GetSelections(ExtendedMetadata metadata)
    {
        yield return new SelectItem 
        { 
            Text = "Existing Page Type One", 
            Value = typeof(ExistingPageOne).FullName 
        };
        yield return new SelectItem 
        { 
            Text = "Existing Page Type Two", 
            Value = typeof(ExistingPageTwo).FullName 
        };

        if (FeatureFlagProvider.IsFeatureFlagActive(FeatureFlagProvider.FeatureOne))
        {
            yield return new SelectItem 
            { 
                Text = "Feature One Page", 
                Value = typeof(FeatureOnePage).FullName 
            };
        }

        if (FeatureFlagProvider.IsFeatureFlagActive(FeatureFlagProvider.FeatureTwo))
        {
            yield return new SelectItem 
            { 
                Text = "Feature Two Page", 
                Value = typeof(FeatureTwoPage).FullName 
            };
        }
    }
}

Finally I needed to prevent the new scheduled jobs for the new integrations from running in higher environments where the feature would be turned off. Again the same FeatureFlagProvider was checked and if the feature was disabled, the scheduled job would immediately exit.

[ScheduledPlugIn(DisplayName = "[Feature One] Content Sync", ...)]
public class FeatureOneSyncScheduledJob : ScheduledJobBase
{
    public override string Execute()
    {
        if (!FeatureFlagProvider.IsFeatureFlagActive(FeatureFlagProvider.FeatureOne))
        {
            return "Feature One Integration is not currently enabled.";
        }

        var results = SynchronizeContent();

        return results.Any() ? string.Join("<br/>", results) : "Complete.";
    }

    private IList<string> SynchronizeContent()
    {
        // Integration Logic Goes Here
    }
}

The Outcome

By using web.config transforms for higher environments, I was able to then turn features on or off at an environment level. As a result of this simple feature flagging approach, we were able to release our three features separately with features turned on or off across multiple releases to production like so:

Release One
- Feature One was feature complete and enabled on Integration and Preproduction.
- Feature Two was incomplete and enabled on Integration only.
- Feature Three was not yet started.
Release Two
- Feature One was released to all environments.
- Feature Two was incomplete and enabled on Integration only.
- Feature Three was not yet started.
Release Three
- Feature Two was feature complete and enabled on Integration and Preproduction.
- Feature Three was released to all environments.
Release Four
- Feature Two was released to all environments.

What About CMS 12?

There are more options to consider when it comes to CMS 12 and my approach would likely be different for a CMS 12 solution. The first thing I would consider is the use of Microsoft's .NET Feature Management package and extending it's usage in the same way. This package comes with built in support for feature flagging on Routing, Filters and Action Attributes. There is also support for feature filters to activate a feature flag with a given set of constraints and the SDK is open source. In this specific case, the only functionality I would have actually used was the ability to check whether a feature flag was enabled.

Another option to consider is whether your client is an Optimizely Feature Experimentation customer. With Feature Experimentation, your client is able to perform experimentations anywhere within the technical stack, however experimenting on the server side does require a development partnership. With Feature Experimentation the customer could choose when to enable or disable specific feature flags without an actual deployment needing to take place. There is a C# SDK to support this which is also open source.

I am an OMVP and the author and maintainer of Stott Security and Stott Robots Handler for Optimizely CMS 12. You can find all of my content collated on https://www.stott.pro/

LLMS.txt support comes to Stott Robots Handler v5

Fri, 08 Aug 2025 08:22:05 GMT

So you've heard of llms.txt?

If you’ve been keeping an eye on developments in AI and search, you may have heard of llms.txt files. The llms.txt file is a simple, human-readable document designed to help Large Language Models (LLMs) better understand the content and context of your website. It is intended to be a new standard in terms of Generative Engine Optimization (GEO) and its fair to say it is early days. You can see a list of early adopters here: https://llmstxt.site/

The llms.txt file itself should be written in Markdown which is a lightweight formatting language that uses plain text syntax to style content. Markdown is then typically transformed into HTML for presentation for normal consumption, but in this case it happens to be easily parsed by AI.

Why Add llms.txt to the Stott Robots Handler?

The Stott Robots Handler has long been the go-to tool for managing robots.txt content in Optimizely CMS 12. Because the llms.txt concept shares a similar purpose and delivery method (providing machine-readable guidance from a fixed location) it made perfect sense to integrate support for it directly into the same tool. The synergy between the delivery and management of the two file formats allowed me to deliver this functionality swiftly into Stott Robots Handler, allowing you to manage your site’s LLM guidance and crawler instructions all from a single, familiar interface.

The Interface

Existing users of the Stott Robots Handler will find the interface for managing llms.txt content to be immediately familiar. The llms.txt configuration list is nearly identical to that of the robots.txt configuration list. Users can create, modify and delete configurations on a per site / per host basis allowing for flexible variations in instructions for large language models. Unlike the robots.txt, there are no default configurations for each website. The llms.txt content only exists and is only served when it has been defined by the user. If a configuration does not exist for a given domain, then the users will receive a 404 response instead of a default value.

When creating or modifying llms.txt content, a familair modal interface is displayed. The modal interface is larger for managing llms.txt content as there are expectations that this file will be significantly larger than robots.txt. Users can select a site and they can opt between selecting a specific host definition or configuring the llms.txt content to be the default for a website.

Please note that when content is served on the /llms.txt path, the response will be resolved in the following order:

llms content that matches the specific site and host definition
llms content that matches a specific site but is configured as a default
404 response

Getting Started

Stott Robots Handler v5 is available for Optimizely PAAS CMS 12 on nuget.optimizely.com and on nuget.org. You can see the full installation instructions and package information over on github page, including how to configure the addon for use with Opti Id.

Getting started can be as simple as:

// Install the nuget package
dotnet add package Stott.Optimizely.RobotsHandler

// Add this to your services
services.AddRobotsHandler();

// Add this to your configure method
services.UseRobotsHandler();

// Add this to your _ViewImports.cshtml if you are running traditional
@addTagHelper *, Stott.Optimizely.RobotsHandler

You can find all of my content collated on https://www.stott.pro/

Optimizing Content Security Policies to Stay Within HTTP Header Limits

Mon, 04 Aug 2025 13:52:54 GMT

As the Stott Security module continues to gain traction across a growing number of Optimizely CMS solutions, I’ve encountered a broader and often more complex range of Content Security Policies (CSPs). Earlier this year, two separate clients reported that their websites were not responding as intended. After investigation, the root cause wasn’t the application server, it was Cloudflare silently dropping the response due to excessive header size.

In both cases, the Content Security Policy (CSP) contained between 220–250 domain entries. After reviewing and optimizing their policies, I was able to resolve the issue by reducing the size of the CSP by 30–50%. In this article, I’ll share common CSP pitfalls and practical techniques to shrink the size of your Content Security Policy, helping you avoid silent failures and stay within browser and CDN limits.

💡Please note Cloudflare will drop any HTTP response where the combined headers exceed 32KB or a single header exceeds 16KB.

- Cloudflare Docs

1. Simplify the directives being used

In CSP 3, the ability to split script-src into script-src-elem and script-src-attr was added. This was intended to give more control over what could be used in a <script> element vs what could be used in a JavaScript attribute on an element. There was a period of time where if you wanted to use script-src-elem and script-src-attr you still had to produce script-src to support devices and browsers that were not yet CSP 3 compliant. CSP 3 support is now very broad so you can omit script-src. If you still have a lot of sources and there is an overlap of sources for script-src-elem and script-src-attr then you could switch back to using script-src instead.

Larger and very specific:

script-src-elem 'self' 'nonce-r4nd0m' https://www.example.com https://www.elem-only.com;
script-src-attr 'self' https://www.example.com https://www.attr-only.com;

Smaller but less specific:

script-src 'self' 'nonce-r4nd0m' https://www.example.com https://www.elem-only.com https://www.attr-only.com;

💡Tip: the same technique can be used for style-src, style-src-elem and style-src-attr.

2. Keep default-src simple

The default-src directive serves as a fallback for most other Content Security Policy directives. If a directive like script-src or img-src isn't explicitly defined, the browser will fall back to whatever you've set in default-src. To reduce complexity and prevent overly permissive defaults, it's best to keep default-src as tight as possible. Ideally restricted to your own domain or even disabled altogether.

Here are three practical options:

default-src 'none'; Blocks all resources unless explicitly allowed by another directive. This is the most restrictive and secure default.
default-src 'self'; Allows only resources from the current domain. This is a common and safe choice for many sites.
default-src 'self' https://*.mydomain.com; More permissive, this allows resources from your domain and all subdomains. Be cautious: this could include dev, test, or legacy subdomains unless you scope them intentionally.

💡Tip: If you're already specifying individual directives like script-src, style-src, and img-src, you may not need a permissive default-src at all. In that case, consider using 'none' to avoid accidentally allowing fallback behaviors you didn’t intend.

3. Keep base-uri simple

The <base> HTML element defines the base URL for all relative links on a page. The base-uri directive in your Content Security Policy restricts which domains are allowed to be set in this element.

Without this restriction, a malicious actor could modify the <base> element to redirect your users’ traffic to an attacker-controlled domain. For example, an attacker could inject a <base href="https://malicious.site/"> tag, causing all relative links on your page to resolve to the malicious site. This can lead to phishing attacks, misleading users into submitting sensitive information or downloading harmful content.

Because this directive serves a very specific purpose, it’s best to keep it narrowly scoped, typically to one of the following:

base-uri 'self';

or

base-uri 'self' https://*.mydomain.com;

4. Keep frame-ancestors simple

The purpose of frame-ancestors is to restrict which websites can host your CMS site in an iframe. For most Optimizely websites, this should be limited to allowing only the website to frame itself:

frame-ancestors 'self';
or
frame-ancestors 'self'; https://*.mydomain.com;

If you are using Optimizely Web Experimentation then you will also want to allow optimizely.com in order to allow the edit variant interface to work:

frame-ancestors 'self'; https://*.mydomain.com https://*.optimizely.com;

5. Avoid duplicate entries

In the Content Security Policies that I reviewed, I noted that there were multiple instances of the same domain being added twice. Both with and without a trailing slash like so:

script-src 'self' https://www.example.com https://www.example.com/

In a Content Security Policy, these two forms are functionally identical. The trailing slash has no effect when the source is just a host (i.e. no path). Browsers treat both as allowing all content from the origin https://www.example.com.

Simplify your policy by removing duplicates and keeping only one clean version:

script-src 'self' https://www.example.com;

💡Tip: If you're using a CSP source that includes a **path**, then the trailing slash *does* matter and it will limit resources to just those immediately below the path:

- https://example.com/js/ matches /js/app.js
- https://example.com/js matches only /js

6. Check if wildcard subdomains already cover more specific domains

In the reported examples, I found multiple instances of redundant wildcard entries for closely related domains. These examples included:

script-src https://*.consentmanager.net https://*.delivery.consentmanager.net https://*.fls.doubleclick.net https://*.g.doubleclick.net https://*.doubleclick.net

When using a wildcard source like https://*.example.com, it's important to understand how wildcards behave:

https://*.example.com matches any number of subdomain levels, such as cdn.sub.example.com
It does **not** match the apex domain itself (i.e. https://example.com)

Knowing this, we can simplify the content security policy to:

script-src https://*.consentmanager.net https://*.doubleclick.net;

💡 Tip: Always verify that the broader wildcard covers all required sources and doesn't introduce unwanted access. If the root domain (e.g. https://doubleclick.net) is needed, it must still be listed separately. It is not included in a wildcard like *.doubleclick.net.

7. Be cautious with https: and wss: wildcard protocols in source directives

On large, multinational CMS platforms (especially those where editors frequently embed third-party content like donation forms, interactive widgets, or 360° views) Content Security Policies can become hard to maintain. In these cases, it's tempting to use a broad directive like:

frame-src 'self' https:;

This allows any content to be iframed as long as it's served over HTTPS, which can be a pragmatic short-term solution when content sources are constantly changing and difficult to manage. However, this convenience comes with serious trade-offs.

Using https: effectively tells the browser to allow any secure domain and not just trusted partners. That means explicitly listing a domain like https://example.com is redundant, and worse, it also implicitly permits https://malicious.site or any other HTTPS-based domain to load scripts or frames on your site. The same behaviour exists for wss: and http: protocols as well. So do be cautious when using them, or avoid them entirely:

script-src 'self' https: https://example.com;  // Redundant and risky

💡Recommendation: While https: or wss: might seem like a helpful shortcut, it's generally better to define a specific allowlist of trusted domains. Avoid https: and wss: as standalone protocol only sources and instead use fully qualified entries like https://trustedpartner.com to maintain control and minimize your exposure to third-party threats.

8. Audit Your Content Security Policy

Optimizely have previously noted that most websites go an average of five years between rebuilds or rebrands. Over that time, it's common for third-party tools to be added and removed, often through platforms like Google Tag Manager (GTM). Each time a tool is introduced, it typically requires updates to your Content Security Policy to allow scripts, iframes, or connections from new domains.

But what happens when you stop using one of those tools? It's easy to forget to remove those permissions from your Content Security Policy. Over time, this results in bloated, outdated, and potentially less secure policies.

The Stott Security module includes features that make auditing easier:

Navigate to the Stott Security Interface
On the Tools page, export your current security configuration
On the CSP Settings page:
1. Enable "Use Report-Only Mode"
2. Enable "Use Internal Reporting Endpoints"
Begin removing CSP sources you believe may no longer be in use
Monitor the CSP Violations page to confirm whether any legitimate content breaks

If any issues arise, you can simply re-import your saved configuration from the Tools page. Once you're confident that your updated policy is safe, turn off "Use Report-Only Mode" and "Use Internal Reporting Endpoints" to enforce the streamlined policy and to reduce traffic to the webserver caused by CSP reports.

9. Use page specific extensions of the source list

Stott Security has long supported the ability to extend the sources for a Content Security Policy for a specific page. Lets say you do have a website with lots of embedded content and most of this is fairly unique. You can end up bloating your CSP just trying to keep up with the number of embeds. This can be undesirable because you will end up allowing a domain to act on all of your website when it is only required on a single page.

To implement this, your development team or agency partner can implement the IContentSecurityPolicyPage interface either as a CMS editable property or by using code you implement yourself as follows:

public class MyPage : PageData, IContentSecurityPolicyPage
{
    [Display(
        Name = "Content Security Policy Sources",
        Description = "The following Content Security Policy Sources will be merged into the global Content Security Policy when visiting this page",
        GroupName = "Security",
        Order = 10)]
    [EditorDescriptor(EditorDescriptorType = typeof(CspSourceMappingEditorDescriptor))]
    public virtual IList<PageCspSourceMapping ContentSecurityPolicySources { get; set; }
}

or

public class MyPage : PageData, IContentSecurityPolicyPage
{
    public IList<PageCspSourceMapping ContentSecurityPolicySources = this.GetSourceListForPage();
}

When this page is served, the sources and directives specified here are merged into the global content security policy just for the purposes of serving this page.

Changes to Stott Security for Optimizely CMS 12

In order to prevent other cusumers of Stott Security from encountering the same issue with header size limits, I decided that I would add some safety nets for the generated Content Security Policy that would respect Cloudflare's hard 16KB limit. When researching header size limits in general, I observed that the most common recommendation was to keep your headers below 8KB per header to ensure broad compatability.

Starting in version 3.0.2 of Stott Security, CSPs are now intelligently split into multiple CSP Headers if their size approaches the 8KB limit. Since browsers enforce the most restrictive policy among multiple CSP headers, these headers are carefully divided based on directive hierarchy and fallback behavior.

If a header grows beyond 12KB, additional logic is applied to consolidate directives as follows:

script-src-elem and script-src-attr are merged into script-src.
style-src-elem and style-src-attr are merged into style-src.
frame-src, worker-src are merged into child-src.
Any omitted source style directives falling back to default-src are added explicitly, defaulting to 'self' if needed.

As a final safeguard, if the combined size of all CSP headers approaches the 16KB Cloudflare limit, the CSP will not be generated to avoid unexpected failures.

Summary

In order to simplify your Content Security Policy and keep it below the recommend 8KB or Cloudflare's hard 16KB limit consider the following:

Simplify Directive Use
- Use just script-src instead of script-src, script-src-elem and script-src-attr
- Use just style-src instead of style-src, style-src-elem and style-src-attr
- Keep default-src, base-uri and frame-ancestors simple by restricting them just to 'self'
Avoid duplicate entries such as https://www.example.com and https://www.example.com/
Avoid using less specific wildcards (https://*.one.example.com) if there is already a more permissive wildcard (https://*.example.com).
Consider the use of the https: and wss: protocol wildcards very carefully as it allows all domains, not just those you specify.
Audit your Content Security Policy and remove permissions for scripts you are no longer using.
Consider using the page specific extension of the Content Security Policy feature that is part of Stott Security
Consider updating to the latest version of Stott Security today in order to benefit from automatic CSP simplification and to protect your servers from bloated content security policies.

You can find all of my content collated on https://www.stott.pro/

Announcing Stott Security Version 3.0

Wed, 09 Apr 2025 08:23:00 GMT

I'm proud to announce the release of version 3 of Stott Security for Optimizely PAAS CMS. This release has been developed over several months owing to a significant new feature among other quality of life changes.

To see the full release notes you can head over to the discussion page on GitHub.

Permissions-Policy Support

The Permissions-Policy HTTP header (formerly Feature-Policy) allows developers to control which web features and APIs can be used in the browser, and by which origin. This header can improve security and performance by restricting access to sensitive capabilities such as the camera, microphone, and geolocation.

Currently, support for the Permissions-Policy header is mixed. It is well-supported by Chromium-based browsers like Chrome and Edge, but not yet implemented in Firefox or Safari. This naturally raises the question: why invest time in implementing a feature that isn’t universally supported?

According to the latest browser market share data, over 70% of users are on browsers that support this header. Additionally, unsupported browsers simply ignore the header, without causing any issues. While I've yet to see a penetration test flag the absence of this header as a vulnerability, I’m increasingly seeing clients request its inclusion as part of their CMS security requirements.

To support this, I’ve added a new Permissions Policy screen to Stott Security. This interface allows administrators to enable or disable the Permissions-Policy header globally with a single toggle. It also includes a filter bar for quickly narrowing directives by source (URL) or current configuration (e.g., *Disabled*, *Allow None*, *Allow All Sites*, etc.).

Clicking the Edit button for a directive opens a modal where the configuration for that specific directive can be adjusted.

Available options include:

Disabled (Omitted this directive from the policy)
Allow None
Allow All Sites
Allow Just This Website
Allow This Website and Specific Third Party Websites
Allow Specific Third Party Websites

For configurations involving third-party origins, administrators can specify one or more sources. Input is strictly validated to require only protocol and domain. Wildcards are supported, but only as the first segment after the protocol (e.g. https://*.example.com).

All changes to the Permissions Policy are fully audited. Additionally, import/export functionality has been extended to include this new feature.

Small Features

.NET 9 Support

If you're building an Optimizely PaaS solution targeting .NET 9, you can now integrate Stott Security into your project, regardless of whether your solution is headed, headless, or hybrid.

Stott Security leverages Entity Framework for database access and operations such as migrations. Since each major release of Entity Framework often introduces breaking changes aligned with specific .NET versions, the Stott Security package has been updated to include build targets for .NET 6, .NET 8, and .NET 9, each using the appropriate version of Entity Framework under the hood.

Import Settings Tool

In previous versions, the import tool required that CSP, CORS, and Response Headers configurations all be present in the import file. To support backwards compatibility with export files created in version 2.x, validation in version 3.x has been relaxed. Now, settings are only applied if they contain a non-null value. This allows for partial imports; if your import file includes only the CSP configuration, then only the CSP settings will be updated, leaving all other settings unchanged.

X-XSS-Protection Header Warning

The X-XSS-Protection header was originally introduced to instruct browsers to enable their built-in XSS filters, aiming to protect users from cross-site scripting attacks. However, the feature introduced its own set of issues. When configured with X-XSS-Protection: 1; mode=block, malicious actors could exploit it to trigger denial-of-service conditions by injecting scripts that caused legitimate content to be blocked. Alternatively, when simply enabled without blocking (X-XSS-Protection: 1), the header became susceptible to XS-Search attacks this is where an attacker submits crafted XSS payloads to probe for differences in application behavior, potentially exposing sensitive data in applications that would otherwise be considered secure.

Although Stott Security continues to support the X-XSS-Protection header, it should only be set to disabled or omitted entirely. To help guide users, explanatory notes have been added to the UI outlining current best practices.

It’s worth noting that Chromium-based browsers already ignore this header entirely, but some other browsers have yet to follow suit.

Content Security Policy Source Updates

Validation has been enhanced to support the 'inline-speculation-rules' keyword within Content Security Policy (CSP) directives. Speculation rules enable the browser to preload potential navigation targets based on user behavior, improving perceived performance by initiating page loads slightly before user interaction (e.g., clicks).

The CSP Edit Source modal in Stott Security has also been updated to ensure that special keywords such as 'unsafe-inline', 'unsafe-eval', and 'inline-speculation-rules' can only be added to directives where they are valid. These are typically limited to script-src, style-src, and their more specific variants.

CMS Editor Gadget Removed

In version 2, I introduced a CMS Editor gadget that displayed the HTTP headers generated when rendering a specific page. This was intended to support the feature allowing users to extend the Content Security Policy (CSP) for individual content items by specifying additional sources.

However, over time it became clear that this approach had several drawbacks:

The gadget was read-only and appeared automatically for all CMS administrators, even if the page-specific CSP feature wasn't in use. In most cases, this meant the gadget was visible but offered little to no value.
Some developers experienced installation issues with the AddOn. Specifically, in projects where the .csproj excluded the modules\_protected folder; often done to avoid conflicts with other third-party AddOns that add files to the solution multiple times. This resulted in the Stott Security module.config being be removed and not re-added. This in turn causes an error during the start up of the solution that could only be resovled by manually adding the into source control and project.

Given the minimal benefit provided by the gadget and the friction it caused in certain build pipelines, I’ve decided to remove it entirely in version 3.

Obsolete Code Removed

If your solution has been using Stott Security since version 1 then you may find that you are using some obsoleted code. As part of packaging up version 3, the following items have been removed:

SecurityServiceExtensions.AddCspManager(...)
- Replaced by SecurityServiceExtensions.AddStottSecurity(...)
SecurityServiceExtensions.UseCspManager()
- Replaced by SecurityServiceExtensions.UseStottSecurity()
CspReportingViewComponent
- Replaced by Report-Uri and Report-To within the CSP.

Report To Endpoint Fixes

The report-uri directive has been deprecated within the content security policy. The specification for this endpoint was for it to receive a single CSP Report per request with a content type of application/csp-report. With the introduction of the report-to directive, browsers are now meant to send a collection of CSP Reports in an array with a content type of application/reports+json. The intent being to reduce the number of requests being made to any reporting endpoint.

Some browsers appear to have simply started sending their report-uri payload to the report-to endpoint on a per report basis instead of matching the specification. The result is a lot of bad requests being returned and a host of reports being lost. I have updated the report-to endpoint so that it can handle both payloads based on their respective content types.

Get It Now

Stott Security v3.0 is free to use and is available on all the usual nuget feeds:

You can find all of my content collated on https://www.stott.pro/

Opti ID with Secure Cookies And Third Party AddOns

Mon, 09 Dec 2024 13:07:46 GMT

Opti ID has revolutionised access to the Optimizely One suite and is now the preferred authentication method on all PAAS CMS websites that I build. However there are a couple of gotchas that always need to be taken care, outlined below are the solutions to both.

Secure Cookies

Any penetration test that you perform on your website will always advise that your cookies are set to be Secure, HTTP Only with a SameSite mode of Strict. Setting these is simple enough until you realise that the Opti ID cookies are third party and therefore for need a SameSite mode of None. If you have used Microsoft Entra (Formerly Azure AD) as the authentication method for a CMS, then this problem will be very familiar to you.

The challenge we have is:

We don't want all cookies to have a SameSite mode of None
Optimizely cookies are not in our direct control
Writing verbose custom cookie code everywhere is messy

In the following solution I have used the app.UseCookiePolicy(...) method to set all of the cookies to be Secure, Http Only with a default SameSite mode of None by default. I then provide a handler for the OnAppendCookie event that will set cookies to use a SameSite mode of Strict provided they do not match one of the following cookie name patterns:

oid- : These are cookies which Opti ID uses for authentication
.AspNetCore : These are cookies NET Core uses for Authentication

public static IApplicationBuilder UseSecureCookies(this IApplicationBuilder app)
{
    // Set the default cookie policy
    app.UseCookiePolicy(new CookiePolicyOptions
    {
        HttpOnly = HttpOnlyPolicy.Always,
        Secure = CookieSecurePolicy.Always,
        MinimumSameSitePolicy = SameSiteMode.None,
        OnAppendCookie = context =>
        {
            if (!context.CookieName.StartsWith("oid-") &&
                !context.CookieName.StartsWith(".AspNetCore"))
            {
                // Any cookie that is not an authentication cookie should have a SameSite mode of Strict
                context.CookieOptions.SameSite = SameSiteMode.Strict;
            }
        }
    });

    return app;
}

Configuring AddOns To Use Opti ID

In Optimizely CMS 11, dependency setup relied upon Initialisation Modules. In CMS 12 and .NET core, all of this is now handled in Startup.cs. Some commonly used AddOns include a service extension that is intended to be consumed within a startup.cs. Some go as far as to provide a custom authorization policy so that you can customise access to the AddOn to specific roles rather than granting full admin access to the CMS for users who just need that functionality. Here are a few AddOns and the roles that you might want to use:

Stott Security - CmsAdmins, SecurityAdmins, DataAnalytics
Stott Robots Handlers - CmsAdmins, SeoAdmins
Geta NotFound Handler - CmsAdmins, SeoAdmins
Geta Sitemaps - CmsAdmins, SeoAdmins

Here is the challenge: These modules will not allow you to access them when using Opti Id unless you specify the Opti ID SchemeName in the authorizarion policy for each AddOn.

Take this configuration for the Stott Security AddOn as an example; this AddOn allows you to segment the data for the AddOn into a separate database and it allows you to define an authorization policy. In this scenario it is a simple matter of making sure the Opti ID Scheme Name is added to the policy:

public static IServiceCollection AddSecurityAddOn(this IServiceCollection services)
{
    services.AddStottSecurity(
        options =>
        {
            options.ConnectionStringName = "EPiServerDB";
        },
        authorization =>
        {
            authorization.AddPolicy(CspConstants.AuthorizationPolicy, policy =>
            {
                // Use the Opti ID scheme Name
                policy.AddAuthenticationSchemes(OptimizelyIdentityDefaults.SchemeName);
                policy.RequireRole(Roles.CmsAdmins, "SecurityAdmins");
            });
        });

    return services;
}

Service Collection Extension Pattern

In both of these examples above, I am following the Service Collection Extension Pattern. I highly recomment this pattern as it allows you to modularise your configuration code and to keep your startup.cs clean and easy to understand or rearrange.

You can find all of my content collated on https://www.stott.pro/

Creating an Optimizely Addon - Best Practices

Thu, 24 Oct 2024 12:32:06 GMT

In Part One, Part Two and Part Three, I have outlined the steps required to create an AddOn for Optimizely CMS, from architecture to packaging at as a NuGet package. In this part I will be covering some best practices that will help you succeed as an AddOn developer. You can view examples from across this series within the this Optimizely AddOn Template that I have been creating.

Unit Tests

As a solo developer managing multiple AddOns, my ability to release updates regularly relies heavily on having extensive unit tests. For instance, Stott Security includes over 1,500 unit tests that run whenever a pull request is made to merge a feature branch into the develop branch. This level of coverage ensures that functionality remains consistent across releases.

As well as writing unit tests for your business logic, you can also write additional unit tests that validate the security of your controllers. I would consider adding these tests to be an essential part of ensuring the security of your system as they ensure the following:

Controller actions explicitly allow only the intended HTTP methods, ensuring endpoints respond only to the correct verbs.
Controller actions are secured with the Authorization attribute or marked with AllowAnonymous if security isn’t required. This enforces clear security requirements for each endpoint.
Controller actions are defined with specific routes, preventing conflicts with other modules or the consuming application’s routing.

[TestFixture]
public sealed class ControllerStandardsTests
{
    [Test]
    [TestCaseSource(typeof(ControllerStandardsTestCases), nameof(ControllerStandardsTestCases.PageControllerActionTestCases))]
    public void ControllersShouldHaveHttpMethodAttributes(string controllerName, string methodName, MethodInfo methodInfo)
    {
        // Act
        var hasHttpMethodAttribute = methodInfo.GetCustomAttributes(typeof(HttpMethodAttribute)).Any();

        // Assert
        // Controllers should only respond in intended Verbs and should respond with method not allowed on unintended verbs.
        // This will prevent posting of malicious payloads to methods not intended to retrieve of deal with these payloads.
        // First raised by a penetration test where an attempt to post files to a content end point returned a 200 instead of a 405
        Assert.That(hasHttpMethodAttribute, Is.True, $"{controllerName}.{methodName} should be decorated with a http method attribute.");
    }

    [Test]
    [TestCaseSource(typeof(ControllerStandardsTestCases), nameof(ControllerStandardsTestCases.PageControllerActionTestCases))]
    public void ControllerMethodsShouldEitherHaveAuthorizeOrAllowAnonymousAttributes(string controllerName, string methodName, MethodInfo methodInfo)
    {
        // Act
        var hasAuthorizeAttribute = methodInfo.GetCustomAttributes(typeof(AuthorizeAttribute)).Any();
        var hasAllowAnonymousAttribute = methodInfo.GetCustomAttributes(typeof(AllowAnonymousAttribute)).Any();
        var controllerHasAuthorizeAttribute = methodInfo.DeclaringType?.GetCustomAttributes(typeof(AuthorizeAttribute)).Any() ?? false;

        var hasAttribute = hasAuthorizeAttribute || hasAllowAnonymousAttribute || controllerHasAuthorizeAttribute;

        // Assert
        // Controller actions should be protected with an Authorization attribute or an intentional AllowAnonymous attribute.
        // This will ensure your controllers are secure by default and that you have to explicitly allow anonymous access.
        Assert.That(hasAttribute, Is.True, $"{controllerName}.{methodName} should be decorated directly or indirectly with an Authorize or AllowAnonymous attribute.");
    }

    [Test]
    [TestCaseSource(typeof(ControllerStandardsTestCases), nameof(ControllerStandardsTestCases.PageControllerActionTestCases))]
    public void ControllerMethodsShouldHaveRouteAttributes(string controllerName, string methodName, MethodInfo methodInfo)
    {
        // Act
        var hasRouteAttribute = methodInfo.GetCustomAttributes(typeof(RouteAttribute)).Any();
        var controllerHasRouteAttribute = methodInfo.DeclaringType?.GetCustomAttributes(typeof(RouteAttribute)).Any() ?? false;

        var hasAttribute = hasRouteAttribute || controllerHasRouteAttribute;

        // Assert
        // Controller actions should have a fixed route attribute so as to not have clashes with routes declared by other modules.
        Assert.That(hasAttribute, Is.True, $"{controllerName}.{methodName} should be decorated directly with a Route attribute.");
    }
}

These tests use a Test Case Source method which uses reflection to identify all controllers within your solution. This means as you add new controllers, you will not forget to secure them.

public static class ControllerStandardsTestCases
{
    public static IEnumerable<TestCaseData> PageControllerActionTestCases
    {
        get
        {
            var assembly = Assembly.GetAssembly(typeof(SettingsLandingPageController));

            if (assembly == null)
            {
                yield break;
            }

            var controllers = assembly.GetTypes()
                                      .Where(t => (t.BaseType?.Name.StartsWith("Controller") ?? false)
                                               || (t.BaseType?.Name.StartsWith("BaseController") ?? false))
                                      .ToList();

            foreach (var controller in controllers)
            {
                var actions = controller.GetMethods()
                                        .Where(x => x.DeclaringType == controller && x.IsPublic)
                                        .ToList();

                foreach (var methodInfo in actions)
                {
                    if (methodInfo.ReturnType == typeof(IActionResult) || methodInfo.ReturnType == typeof(Task<IActionResult>))
                    {
                        yield return new TestCaseData(controller.Name, methodInfo.Name, methodInfo);
                    }
                }
            }
        }
    }
}

Test System

If you have followed this series to create an AddOn, you'll note that the sample site for developing the AddOn does not use nuget to consume the AddOn code and instead uses a project reference. This will allow you to develop efficiently, however it does mean that you are not testing your code in a production-like manner.

Create a separate repository with it's own Optimizely CMS solution. Import your AddOn as a NuGet package directly into this solution. This will allow you to test your AddOn in the same way that another developer will be experiencing your AddOn for the first time. If you are able to generate a developer cloud license on EPiServer License Centre, then I would recommend you deploy this test system into an Azure WebApp running on Linux with .NET 6.0 or 8.0 so that you can validate your AddOn inside of a deployed environment.

As part of my go live cycle I perform the following:

Create a beta version of my nuget package
Upload the beta package to nuget.org
Update my test system to use the beta package
Deploy my test system to Azure
Delist my beta version on nuget.org
Test my AddOn in my test system
Create a production version of my nuget package
Upload the production package to nuget.optimizely.com
Wait for the package to be approved by Optimizely's QA team
Upload the production package to nuget.org
Announce the release

Performance and Caching

Optimizely's default settings for Azure SQL Server typically support around 120 simultaneous database connections, suggesting the use of an S2 SQL Database (or equivelent) for production. This configuration, combined with efficient caching by the Optimizely CMS code allows smaller databases to perform effectively for websites with high traffic.

If you're using Microsoft Entity Framework, be aware that each DbContext instance opens a new database connection. Failing to manage these connections can lead to server instability due to connection limits. Therefore, it's advisable to follow Optimizely's approach by extensively using caching. To implement this, consider the following steps:

Use a Custom Cache Wrapper that consumes Optimizely's ISynchronizedObjectInstanceCache and uses it's own master key to allow you to purge your cache effectively.
Inject your DbContext as a scoped object to limit the number of instances to 1 per request.
Lazy Load dependencies that require a Db Context so that they are not instantiated if not consumed.
Handle data loading in the following order:
- Attempt to retrieve and return data from cache first.
- Attempt to retrieve and return data from the database second.
  - Push the data into a cache before returning it.

The following is an example of a cache wrapper that consumes the ISynchronizedObjectInstanceCache.

public sealed class CacheWrapper : ICacheWrapper
{
    private readonly ISynchronizedObjectInstanceCache _cache;

    private const string MasterKey = "My-OptimizelyAddOn-MasterKey";

    public CacheWrapper(ISynchronizedObjectInstanceCache cache)
    {
        _cache = cache;
    }

    public void Add<T>(string cacheKey, T? objectToCache)
        where T : class
    {
        if (string.IsNullOrWhiteSpace(cacheKey) || objectToCache == null)
        {
            return;
        }

        try
        {
            var evictionPolicy = new CacheEvictionPolicy(
                TimeSpan.FromHours(12),
                CacheTimeoutType.Absolute,
                Enumerable.Empty<string>(),
                new[] { MasterKey });

            _cache.Insert(cacheKey, objectToCache, evictionPolicy);
        }
        catch (Exception exception)
        {
            // Add logging here
        }
    }

    public T? Get<T>(string cacheKey)
        where T : class
    {
        return _cache.TryGet<T>(cacheKey, ReadStrategy.Wait, out var cachedObject) ? cachedObject : default;
    }

    public void RemoveAll()
    {
        try
        {
            _cache.Remove(MasterKey);
        }
        catch (Exception exception)
        {
            // Add logging here
        }
    }
}

In order to use Lazy Loaded dependencies, you first need to define the Lazy variant within your service extension method:

services.AddScoped<IMyDataContext, MyDataContext>();
services.AddScoped<Lazy<IMyDataContext>>(provider => new Lazy<IMyDataContext>(() => provider.GetRequiredService<IMyDataContext>()));

You can then declare your dependencies as Lazy in your constructors and consume them as per the example below. In this scenario, the IMyDataContext is instantiated once, but that is deferred until it is used by the GetData() method:

internal sealed class MyRepository : IMyRepository
{
  private readonly Lazy<IMyDataContext> _context;

  public MyRepository(Lazy<IMyDataContext> context)
  {
    _context = context;
  }

  public async Task<IList<string>> GetData()
  {
    return await _context.Value.MyData.ToListAsync();
  }
}

A service can then consume both the IMyRepositoryand ICacheWrapper and make performant calls to retrieve data that has not changed. In the following example we attempt to retrieve the data from the cache first, then if it is null or empty we then attempt to load the data from the repository, push that data into cache before returning it. If the Delete method is called within the service, we call RemoveAll() on the cache wrapper to invalidate cache entries based on a master key:

internal sealed class MyService : IMyService
{
  private readonly IMyRepository _repository;
  private readonly ICacheWrapper _cache;
  private const string CacheKey = "Unique.Cache.Key";

  public MyService(IMyRepository repository, ICacheWrapper cache)
  {
    _repository = repository;
    _cache = cache;
  }

  public async Task<IList<string>> GetDate()
  {
    var data = _cache.Get<IList<string>>(CacheKey);
    if (data is not { Count: >0 })
    {
      data = await _repository.GetData();
      _cache.Add(CacheKey, data);
    }

    return data;
  }

  public async Task Delete(string data)
  {
    await _repository.Delete(data);

    _cache.RemoveAll();
  }
}

Note how we don't need to lazy load the repository as the database context is already lazy loaded by the repository itself, however you may choose to lazy load the repository as it is not used if the cache is populated.

Summary

Ensure all key business logic is covered by a unit test.
Use unit tests to enforce standards such as:
- Controller actions have correct HTTP method attributes.
- Controllers have secure actions with authorization or explicit anonymous access.
- Controllers have defined routes to avoid conflicts.
Test your AddOn in a separate production-like environment.
Use caching to optimize database access.
Implement lazy loading and scoped dependencies.

You can find all of my content collated on https://www.stott.pro/

Creating an Optimizely Addon - Packaging for NuGet

Mon, 16 Sep 2024 08:17:11 GMT

In Part One and Part Two of this series; I covered topics from having a great idea, solution structure, extending the menus and adding gadgets to the editor interface. In this part I will be covering the challenges of creating and submitting your AddOn as a NuGet package into the Optimizely NuGet feed. You can view examples from across this series within the this Optimizely AddOn Template that I have been creating.

Defining What to Package

Each project within your solution must be created as a NuGet package if it is deemed to be the primary project or a dependency for the primary project. Consider the following solution as an example:

MySolution.sln
- MyAddOn.Admin.csproj
- MyAddOn.Core.csproj
- MyAddOn.Test.csproj

In this scenario, the administrator interface for the AddOn is separated from its core shared functionality. This design enables a consuming site to incorporate MyAddOn.Admin into their Optimizely website project and to reference MyAddOn.Core within any project in their solution structure. Consequently, MyAddOn.Admin has a direct dependency on MyAddOn.Core. To publish MyAddOn.Admin as a NuGet package, MyAddOn.Core must also be published as a NuGet package. It should be noted that MyAddOn.Admin only requires MyAddOn.Core as a project dependency during development; this dependency will be converted into a package dependency during the packaging process.

Defining NuGet Properties

If you are using Visual Studio, right click on the project you want to package and select properties to show the project properties screen. Under the Package section you can define all of the properties for your NuGet package.

I would recommend you complete the following:

Package Id : This will need to be globally unique name within nuget.org and nuget.optimizely.com. If you use the $(AssemblyName) variable, then this will match the name of the project.
Title : Visual Studio describes this as the name of the package used in UI displays such as Package Manager, but this largely does not get used.
Package Version : This should be a semantic version number with three or four parts and an optional alpha or beta tag. For Example:
- 1.0.0
- 1.0.0.0
- 0.1.1-alpha
- 0.2.2.0-beta
Authors : This should contain the names of the primary people who will own the AddOn / Repository.
Company : This should contain the name of the business that is behind creating the Addon. If this is individually owned, then seting this to $(Authors) will mirror the value from the Authors property.
Description : This should be a short description about your Addon, this will be visible within the NuGet package feed and within the Plugin Manager screen within Optimizely CMS.
Copyright : This should contain the name of the owner and the year. You get copyright protection automatically when creating software and you do not have to apply or pay a fee. There isn’t a register of copyright works in the UK. There are however organisations which will provide extra protection for a fee for validating your copyright. You can read more about copyright here: How copyright protects your work. It is however worth you performing your own research into the matter within the country you live in.
Project Url : This should point either to the repository for your Addon or an appropriate project page. Developers will use this to find out more about your Addon or to report issues that may need resolving.
Readme : I have set this to the readme.md for my repositories, this will be visible to developers within the NuGet platform.
- Do ensure assets such as images have absolute paths as this readme will be visible outside of the context of your repository and relative paths will result in images not being found.
Repository Url : This should point to the repository for your Addon, assuming that your Addon is Open Source.
Tags : This is a delimited set of tags that make your package easier to find within the NuGet feeds.
License File : This should reference the license within your repository. Careful consideration should be given to the type of license for your AddOn. Certain licenses may require your users to make their code open source to utilize your package, so think carefully about the permissiveness or restrictiveness of your license. It is noteworthy that some highly popular AddOns employ an MIT or Apache license.
- I am utilizing an MIT license due to its permissive nature and lack of warranty. While I do engage with my users and address any issues that are raised, my AddOns are free and are maintained in my free time.
Require License Acceptance : If you tick this, the consumer will have to accept the license as they install the package. If you are using an MIT license, you may want to tick this to encourage the consumer to accept the warranty free nature of your AddOn.

If you are using Visual Studio Code instead of Visual Studio, then you can edit the .csproj directly and add the package properties directly as XML values at the top of the csproj file. You can also add these properties into a .nuspec instead, when you package your project, the values from the .csproj and .nuspec are merged into a new .nuspec that is contained in the root of the compiled .nupkg file. I personnally prefer to put the NuGet properties directly into the .csproj.

<Project Sdk="Microsoft.NET.Sdk.Razor">
  <PropertyGroup>
    <TargetFrameworks>net6.0;net8.0</TargetFrameworks>
    <AddRazorSupportForMvc>true</AddRazorSupportForMvc>
    <Version>1.1.0.0</Version>
    <RepositoryUrl>https://example.com/</RepositoryUrl>
    <PackageProjectUrl>https://example.com/</PackageProjectUrl>
    <PackageLicenseFile>LICENSE.txt</PackageLicenseFile>
    <Authors>Your Name</Authors>
    <Description>Your Package Summary</Description>
    <Copyright>Your Name 2024</Copyright>
    <PackageTags>TagOne TagTwo</PackageTags>
    <PackageRequireLicenseAcceptance>true</PackageRequireLicenseAcceptance>
    <RepositoryType>git</RepositoryType>
    <PackageReadmeFile>README.md</PackageReadmeFile>
    <AssemblyVersion>1.1.0.0</AssemblyVersion>
    <GeneratePackageOnBuild>True</GeneratePackageOnBuild>
    <PackageReleaseNotes>A short release summary.</PackageReleaseNotes>
    <Nullable>enable</Nullable>
    <Title>Package Name</Title>
  </PropertyGroup>

NuGet Package Structure

A NuGet Package is simply a zip file containing a structured set of files. If you rename a .nupkg to a .zip, you can extract it and explore it's structure. This will have a structure similar to the following:

package
- services
  - metadata
    - core-properties
build
- project.name.targets
contentFiles
- additional.file.txt
lib
- net6.0
  - my.project.dll
- net8.0
  - my.project.dll
- my.project.nuspec
_rels
[Content_Types].xml
readme.md
license.txt

Folders such as build, contentFiles and the target folders under lib will vary depending on your code and deployable files. The readme.md and license.txt files referenced in your .csproj or .nuspec are copied to the root of the NuGet package.

Packaging for Multiple Frameworks

.NET Core is backwards compatible, meaning that if you build your package for .NET 6, it can be installed into .NET 6, 7, and 8. For most AddOns, compiling directly for .NET 6 ensures maximum compatibility.

However, there may be instances where you need to compile your application in multiple framework versions. For example, if you are using Entity Framework and Migrations, there is a breaking change between .NET 6 and .NET 8. Fortunately, no code changes are required, but you will need to set your dependencies separately for .NET 6 and .NET 8. To accomplish this, you must make two modifications.

Change the TargetFramework node in your .csproj to be TargetFrameworks and separate your target frameworks with a semicolon. e.g. net6.0;net8.0.
Add a separate ItemGroup per framework version to contain framework specific dependencies and add a condition to the ItemGroup to target the specific framework. e.g. Condition="'$(TargetFramework)' == 'net6.0'".

<Project Sdk="Microsoft.NET.Sdk.Razor">
  <PropertyGroup>
    <TargetFrameworks>net6.0;net8.0</TargetFrameworks>
    <AddRazorSupportForMvc>true</AddRazorSupportForMvc>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup Condition="'$(TargetFramework)' == 'net6.0'">
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="6.0.6" />
  </ItemGroup>

  <ItemGroup Condition="'$(TargetFramework)' == 'net8.0'">
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="8.0.1" />
  </ItemGroup>

This will double the size of your NuGet package as it will contain separate folders for each target framework containing your code compiled for that framework.

Packaging Additional Files For The Protected Modules Folder

If your package contains an IFrameComponent or other files needed to extend the Editor Interface. A module.config file and those files will need to be deployed to the modules/_protected/my_project folder within the target website.

First you will need to tell the .csproj file that we want to copy these files into the contentFiles folder of the NuGet package. This is as simple as setting the build output for those files to be None and to set the PackagePath to be inside of the contentFiles folder.

<ItemGroup>
  <None Include="module.config">
    <Pack>true</Pack>
    <PackagePath>contentFiles\module.config</PackagePath>
  </None>
</ItemGroup>

You will then need to create a .targets file that instructs the NuGet package installer how to handle those files. The example below is taken straight from my own Addons where I am doing the same thing.

The ItemGroup tells the .targets file where the specific files are within the NuGet package structure. The $(MSBuildThisFileDirectory) variable in this case is a reference to the directory the .targets file sits in. As this is in a build folder, I have used the $(MSBuildThisFileDirectory) variable in combination with the relative path to my module.config file.

The Target node is then performing an action that is configured to execute on BeforeBuild. This then performs a Copy action that will take my module.config file from the contentFiles folder in the nuget package to the modules\_protected\my_project folder within the target website. This means that when you first install the package, the module.config file and folder will not exist within the protected modules folder. When you first build the solution they will be copied into this location.

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
  <ItemGroup>
    <MyFiles Include="$(MSBuildThisFileDirectory)..\contentFiles\module.config" />
  </ItemGroup>
  
  <Target Name="CopyFiles" BeforeTargets="BeforeBuild">
        <Copy SourceFiles="@(MyFiles)" DestinationFolder="$(MSBuildProjectDirectory)\modules\_protected\my_project\" />
    </Target>
</Project>

In order to make sure the .targets file can be executed, we also need to make sure that it is copied into the NuGet package file. This is as simple as editing your .csproj file and configuring the build output for the .targets file to be None and to set the PackagePath to be inside of the build folder.

<ItemGroup>
  <None Include="msbuild\copyfiles.targets">
    <Pack>true</Pack>
    <PackagePath>build\$(MSBuildProjectName).targets</PackagePath>
  </None>
</ItemGroup>

Submitting Your Package

Before submitting your AddOn to the Optimizely NuGet package feed, it is essential to ensure that your package installs successfully both in your local environment and within a CI/CD pipeline. To expedite this process, consider publishing your package as an alpha or beta build to nuget.org first. After publishing, your package will be indexed and available for retrieval within a few minutes.

To designate your package as an alpha or beta release, you should modify the version property within your `.csproj` file to include a trailing -alpha or -beta. NuGet will automatically recognize this as a pre-release version and will generally filter these versions out by default. Developers can view these pre-release versions by selecting the option to display pre-release versions within their IDE’s NuGet package tool.

<Version>2.0.0.2-beta</Version>

Upon publishing the alpha or beta version of your package to nuget.org and confirming its successful installation both locally and in a CI/CD pipeline, you will be prepared to submit the live version of your package to Optimizely.

Ensure that you have an Optimizely World account. You can create a new account by visiting Optimizely World and following the registration link located in the top right corner. This account will also provide access to the Optimizely NuGet feeds. Optimizely maintains two NuGet feeds:

https://nuget.optimizely.com (v2 NuGet feed)
https://api.nuget.optimizely.com (v3 NuGet feed)

Packages uploaded to the v2 NuGet feed are automatically synchronized to the v3 NuGet feed. Therefore, it is advisable to upload your packages to the v2 NuGet feed. Once Optimizely receives your package, it will undergo an approval process conducted by Optimizely's QA team. During this process, the QA team will verify that your AddOn functions correctly with the CMS. Including test guidance in the readme for your repository can be very beneficial for the QA team. This review process may take one or more business days, and there is currently no feedback mechanism to inform you of the status or outcome of the testing. You may periodically check the NuGet feed to determine if your package has been accepted. Given that Optimizely validates all packages uploaded to their NuGet feed, it is recommended to download AddOn updates directly from Optimizely and distribute your own package in this manner. Should you need to release a hotfix promptly, you may consider uploading it to nuget.org.

It is advisable to upload your package to nuget.org at least once in addition to the Optimizely NuGet feed. This ensures that the package name is reserved on nuget.org, avoiding potential conflicts in package names across the main feeds that could affect your consumers.

Please note that as of the time of writing, there was an issue with packages uploaded directly to the v3 NuGet feed not being synchronized back to the v2 NuGet feed. Until this issue is resolved, the Upload link on the v3 NuGet feed redirects users to the v2 NuGet feed. Optimizely is actively working to resolve this issue.

Summary

Build your package for .NET 6 for maximum compatability
- Build your package for both .NET 6 & 8 if you have compatability issues between both frameworks.
Use a Razor Class Library so you can package your UI and C# code together.
Think very carefully about which license you will use for your package.
Use a build targets file to put files into specific folders within a consuming application.
Test your package installs and works as an alpha/beta on nuget.org before submitting to the Optimizely NuGet feed.
Upload your package to nuget.optimizely.com when it is ready.

You can find all of my content collated on https://www.stott.pro/

Creating an Optimizely CMS Addon - Adding an Editor Interface Gadget

Fri, 30 Aug 2024 15:43:33 GMT

In Part One of this series, I covered getting started with creating your own AddOn for Optimizely CMS 12. This covered what I consider to be an ideal solution structure, best practices for your JavaScript and Styles, extending the menu interface and authentication. In Part Two, I will be covering adding an additional editor interface gadget. You can view examples from across this series within the this Optimizely AddOn Template that I have been creating.

Adding a Gadget to the Editor Interface

You can easily turn a standard MVC Controller into a CMS Editor Gadget by decorating it with the [IFrameComponent] attribute. The primary properties of the attributes are as follows:

Title : The name of the gadget, visible in the Gadget selector.
Description : A brief description of the gadget, also visible in the Gadget selector. It is recommended to keep this to one sentence.
Categories : The appropriate category for the gadget. For content-specific gadgets, this should be set to "content" or "cms"
Url : The route corresponding to your controller action.
PlugInAreas : The location within the system where the component should be made available.
ReloadOnContextChange : Enables the UI to reload the gadget each time a different content item is selected within the CMS interface.

Optimizely will automatically identify these controllers and use the properties of the [IFrameComponent] attribute to populate the Add Gadgets window within the CMS Editor interface:

When the editor interface loads your gadget, it will include an id query string parameter containing a versioned content reference in string format (e.g., 123_456). The number on the left represents the permanent identity of the content, while the number on the right denotes the specific version of that content item. This information can be used to load the specific version of a content item and incorporate it into your gadget's model.

Below is an example of the minimum required setup for your controller. Note the inclusion of the [Authorize] attribute on the controller and the [HttpGet] attribute on the action. These ensure that the user is authenticated and that the interface cannot be accessed using an unexpected HTTP verb. Note how I use the id query string parameter to load the specific version of the page the user has selected within the CMS Editor interface to provide context specific information to the user.

[Authorize(Policy = OptimizelyAddOnConstants.AuthorizationPolicy)]
[IFrameComponent(
    Url = "/optimizely-addon/gadget/index/",
    Title = "Example Gadget",
    Description = "An example gadget for the CMS Editor Interface.",
    Categories = "content",
    PlugInAreas = "/episerver/cms/assets",
    MinHeight = 200,
    MaxHeight = 800,
    ReloadOnContextChange = true)]
public sealed class GadgetController : Controller
{
    private readonly IContentLoader _contentLoader;

    public GadgetController(IContentLoader contentLoader)
    {
        _contentLoader = contentLoader;
    }

    [HttpGet]
    [Route("~/optimizely-addon/gadget/index")]
    public IActionResult Index()
    {
        var model = new GadgetViewModel
        {
            Page = GetPageData(Request),
            ContentId = Request.Query["Id"].ToString()
        };

        return View("~/Views/OptimizelyAddOn/Gadget/Index.cshtml", model);
    }

    private PageData? GetPageData(HttpRequest request)
    {
        var contentReferenceValue = request.Query["Id"].ToString() ?? string.Empty;
        if (string.IsNullOrWhiteSpace(contentReferenceValue))
        {
            return null;
        }

        var contentReference = new ContentReference(contentReferenceValue);
        if (_contentLoader.TryGet<PageData>(contentReference, out var pageData))
        {
            return pageData;
        }

        return null;
    }
}

In the gadget I developed for Stott Security, I focus exclusively on rendering a preview of the security headers for the currently selected page. I added this feature as Stott Security supports extending the Content Security Policy for any given page. To ensure the user can be clear on the context for the header preview, I load the page using the same helper method and add it to the view model for my Gadget.

Extending IFrameComponent

When your user logs into the CMS, they will be given your new Gadget by default. Now you can ensure that only specific roles have access to the Gadget by setting the AllowedRoles property within the [IFrameComponent] declaration. If your AddOn allows the developer to define a custom security policy for accessing your module, you cannot simply specify the roles within the attribute. For next version of the Stott Security AddOn, I have created a SecureIFrameComponentAttribute that inherits the IFrameComponentAttribute and dynamically resolves the roles that are allowed access to the Gadget based on that security profile.

[AttributeUsage(AttributeTargets.Class)]
public sealed class SecureIFrameComponentAttribute : IFrameComponentAttribute
{
    public SecureIFrameComponentAttribute() : base()
    {
        try
        {
            var authorizationOptions = ServiceLocator.Current.GetService(typeof(IOptions<AuthorizationOptions>)) as IOptions<AuthorizationOptions>;
            var policy = authorizationOptions?.Value?.GetPolicy(CspConstants.AuthorizationPolicy);
            var roles = policy?.Requirements?.OfType<RolesAuthorizationRequirement>().ToList();
            var roleNames = roles?.SelectMany(x => x.AllowedRoles).ToList() ?? new List<string> { Roles.WebAdmins, Roles.CmsAdmins, Roles.WebAdmins };

            AllowedRoles = string.Join(',', roleNames);
        }
        catch(Exception)
        {
        }
    }
}

Telling the CMS Editor Interface About Our AddOn

There are two steps to enable the Editor Interface to recognize our AddOn. The first step is to declare our assembly in a module.config file. Personnally this doesn't feel like it should be a requirement as all of the information is provided in the [IFrameComponent] attribute, though it appears that a validation during application startup mandates that this configuration file exists. I suspect this is a requirement tied to much deeper integrations with the UI. E.g. custom DOJO editor code etc.

Below is an example of a module.config file. Note the inclusion of an Authorization Policy as an attribute of the module node; this should correspond to the policy required by your AddOn. Additionally, ensure that the full name of the assembly containing your gadget is listed within the assemblies node.

<?xml version="1.0" encoding="utf-8" ?>
<module loadFromBin="true" clientResourceRelativePath="" viewEngine="Razor" authorizationPolicy="MyAddOn:Policy" moduleJsonSerializerType="None" prefferedUiJsonSerializerType="Net">
  <assemblies>
    <add assembly="MyAddOnAssemblyName" />
  </assemblies>

  <clientModule>
    <moduleDependencies>
      <add dependency="CMS" />
    </moduleDependencies>
  </clientModule>
</module>

If you are simply adding a gadget to a specific Optimizely CMS build, the assembly declaration can be included in the module.config file located at the root of your website application. However, in the context of an AddOn, this declaration should be placed within a protected modules folder, using a path such as [MyCmsWebsite]/modules/_protected/[MyAddOn]/module.config. There are some extra steps required to achieve this when creating a NuGet package and I address these in Part Three of this series which is focused entirely on the NuGet package process.

The second step that is needed to inform the CMS of our AddOn is to ensure that is included within the ProtectedModuleOptions. This can be achieved within a service extensions method that you call within your startup.cs as follows:

public static class OptimizelyAddOnServiceExtensions
{
    public static IServiceCollection AddOptimizelyAddOn(this IServiceCollection services)
    {
        services.Configure<ProtectedModuleOptions>(
            options =>
            {
                if (!options.Items.Any(x => string.Equals(x.Name, "MyAddOnAssemblyName", StringComparison.OrdinalIgnoreCase)))
                {
                    options.Items.Add(new ModuleDetails { Name = "MyAddOnAssemblyName" });
                }
            });

        return services;
    }
}

Summary

Add the [IFrameComponent] attribute to your controller to define the gadget for the CMS Editor Interface.
Add the [Authorize] attribute to your controller to secure it.
Add HTTP verb attributes such as [HttpGet] to your controller actions to prevent unexpected access attempts.
Add a module.config file to the [MyCmsWebsite]/modules/_protected/[MyAddOn]/module.config folder so that CMS can validate your gadget assembly.

You can find all of my content collated on https://www.stott.pro/