Blog posts by Thanh-Vi Le

Optimizely Cross-Browser Testing to deprecate on April 25th, 2023

Fri, 24 Feb 2023 16:03:33 GMT

Optimizely will deprecate the Cross-Browser Testing feature on the 25th of April 2023.

As of this writing, cross-browser testing only has 0.54% user adoption rate and 1.38% account adoption rate. In fact, the actual percentages are even lower as these currently include Optimizely employees who emulate custom accounts.

The current version of cross-browser testing is being deprecated which would force Optimizely to migrate to the new version in order to maintain this feature. However, implementing a new version of cross-browser testing requires our developers to spend a disproportionate amount of time building and maintaining the feature; therefore, due to its lack of customer adoption, Optimizely has made the decision to shut it down.

Thank you for trusting Optimizely with providing you the means to create experiences that your customers love. Please contact support@optimizely.com if you need assistance with our Web Experimentation and/or Performance Edge products, and don’t hesitate to use https://feedback.optimizely.com/ so we can continue making (y)our products better.

Track Experiments with Component Trackers – Make Experimentation Seamless

Tue, 14 Feb 2023 17:42:49 GMT

The need for component trackers

Running experimentation programs effectively across organizations can be difficult, especially when running multiple tests at a time – why? Because it entails keeping track of all of them while also managing them efficiently, which has grown to be a pain point for most customers. In fact, not doing this properly would lead to misallocating time and resources, resulting in slower experimentation and fumbled revenue.

This is a particularly hefty and time-intensive task for:

Large organizations that run a high number of tests simultaneously
Customers who are very new at experimentation, need guidance, and don’t yet know how to manage the building blocks of their experiments i.e. their components.

Therefore, Optimizely is hyper-focused on making sure that our product allows all Experiment Collaborators to get a bird’s eye view of where their components are being used.

Experiment Collaborators Need Enhanced Experimentation Coverage

Many questions rise in the process of developing and navigating experiments. To effectively run an experimentation program, Experiment Collaborators need to understand the following:

Where experiments are running and if there are any conflicts that could potentially misconstrue the data
What pages, events, extensions, and audience resources are being used in which experiments
What pages, events, extensions, and audiences can be archived without affecting any workflows

For so long our Optimizely customers could only get insight into these things if they emailed their SA to run a script and bulk archive pages, and if they had no SA support they would manually go through their repository of pages, events, extensions, and audiences to archive those not being used in an experiment. Typically, Experiment Collaborators were tasked with governance and housekeeping, which involved:

Using the right pages, events, extensions, and audiences in experiments that are configured to the organization's definition
Cleaning up unused or outdated components that clutter the UI, as they can cause confusion for users setting up new experiments.

Therefore, Optimizely has come up with a new feature to enhance governance and housekeeping processes: Component Trackers.

The Solution: Introducing Component Trackers

To identify if there are any conflicts with an experiment, or to find out what components an experiment is being used on, we now have the capability within our app to identify the number of active experiments currently using the same components.

At Optimizely, it is our priority to ensure that Experiment Collaborators have all the answers that they need to make effective changes and decisions when it comes to experimentation. This new feature identifies the number of active experiments targeting the same components—making experimentation more efficient and effective. The way that an Experiment Collaborator will see it within Optimizely:

View when you are creating or modifying an experiment in “Implementation”
- The number of experiments a page is currently being used in
- A view that allows a user to dig into which experiments are being used on that specific page

Conclusion

With Component Trackers, the management of experimentation assets will be significantly easier. Experiment Collaborators will be able to identify that the correct Pages, Events, Extensions and Audiences are being used in their experiments and configured properly. Additionally, governance and housekeeping will be substantially more straightforward given the effortless identification of unused experimentation assets. Components Trackers will make experimentation seamless by increasing experiment velocity and performance and providing enhanced management.

Questions? Please contact support@optimizely.com.

Optimizely Web Experimentation to end support for Internet Explorer (IE) in 2023

Tue, 10 Jan 2023 16:25:40 GMT

Optimizely Web Experimentation, including both the Web and Performance Edge products, will no longer support any version of IE on the 15th of April 2023.

Microsoft ended support for IE11, the most recent version of IE, on June 15, 2022. Despite this, the Web and Edge snippets have continued to support IE11 through the end of 2022. The decision to drop support comes from the lack of visitor traffic. As of this writing, IE makes up only about 0.248% of traffic amongst the top 10 most popular browsers that our customers’ visitors use and only 0.247% of our total browser usage, yet our developers spend a disproportionate amount of time testing and resolving support tickets tied to it.

This decision entails that Optimizely will no longer validate that new changes to the Web and Performance Edge snippets work on IE11. We won’t force any specific behaviours based on browsers: users can choose to conditionally load the snippet on supported browsers, or to rely on our variety of custom JS areas rather than our OOTB logic – ensuring their JS will run on any browser.

Note: Optimizely Full Stack also dropped support and stopped testing for Internet Explorer in June 2022.

Take experimentation velocity to the next level with Mutually Exclusive Experiments in Edge

Tue, 15 Nov 2022 17:43:30 GMT

The need for Mutually Exclusive Experiments in Edge

When they visit our customers’ sites, Edge users are likely to see more than one related experiment running simultaneously. This could cause them to behave differently from visitors who only see one of these experiments in isolation. As a result, this difference in behavior has the potential to affect our customers’ results and lead to undesirable interactions between experiments. Experiment collaborators can’t afford to make inaccurate inferences as they need reliable results to understand their customers better: they need to be empowered with the right tools to build relevant experiences and succeed faster.

Furthermore, we want our customers to have the flexibility of running as many experiments as they can, whether they wish to use Web Experimentation or Performance Edge. The development team at Optimizely has worked hard to address this pain point in our Web and Feature Experimentation products, and now we’re bringing it to Performance Edge.

Experiment collaborators need to be confident in their results: introducing Mutually Exclusive Experiments in Edge

With Mutually Exclusive Experiments available in Edge, we will make our customers’ experiments and campaigns mutually exclusive via exclusion groups. Suppose that our customers are running two experiments, but both are in an exclusion group. This means that any visitor who sees Experiment A won’t see Experiment B, and vice versa. For instance, say that Experiment A has a 50% traffic allocation and Experiment B has a 50% traffic allocation. Half of the users will only see Experiment A, and the other half will only see Experiment B.

As a result, no visitor will see both experiments because they are in the same exclusion group, which makes both experiments mutually exclusive. These experiments would be customizable in a way that traffic allocation for experiments or campaigns in an exclusion group can be changed. This prevents customers from collecting unnecessary biases from users’ difference in behaviors when being added to several experiments.

Here's how to create an exclusion group in Edge:

The takeaway

Thanks to Mutually Exclusive Experiments in Edge, avoid inaccurate results that stem from isolating related experiments. Make sure you invest in the right changes and increase experiment velocity by allowing more experiments to be run concurrently with higher confidence in results. Read more about our new Mutually Exclusive Experiments feature here and here. Note: Mutually Exclusive Experiments for Edge will be rolled out gradually over Q4, and is expected to be out 100% by early December 2022.

Questions?

Please contact support@optimizely.com if you need assistance with our Web Experimentation and/or Performance Edge products. If you’d like to follow our development updates, feel free to subscribe to our RSS feed by clicking the corresponding icon below.

Create awareness with Concurrent Editing – Enable smooth cross-collaboration within your team

Fri, 12 Aug 2022 12:47:45 GMT

The need for Concurrent Editing

Cross-collaboration within organizations is a must in order to be successful and complete projects as efficiently as possible. However, when multiple people concurrently edit the same experiment or variation unknowingly, it may cause constant overwriting between users, thus making teamwork inefficient and confusing. In fact, failing to collaborate efficiently can lead to misallocated time and resources, resulting in slower experimentation and missed opportunities.

Efficient cross-collaboration can be particularly complex for:

High-activity companies with numerous collaborators wanting to work on the same experiment at the same time
Low-activity companies which haven’t yet established adequate internal communications

For example, experiment collaborators might work on an experiment for several hours only to see their changes being overwritten by someone else’s edits. This is a pain that has been known to be frustrating to users. Therefore, Optimizely is committed to making sure that our Web Experimentation product allows multiple experiment collaborators to edit an experiment simultaneously by informing active users what others are working on in real time. By collaborating live together, users will save time and have more visibility on others’ contributions.

Experiment Collaborators Need To Know What’s Going On At All Times: Introducing Concurrent Editing Awareness

To solve this, Optimizely’s teams came up with a feature that will display a banner notification to inform active users of other active users who are viewing/editing the same project.

The way that an experiment collaborator will see it within Optimizely:

View when “More info” is clicked on the experiment details screen
- Get warning message about potential overwriting
- See who is viewing/editing the page and their respective email address

The warning banner can be dismissed by clicking “Close”; however, it may reappear in 2 instances: (1) when a new user starts editing the same experiment mid-session and (2) when a session ends and restarts (every new session/page refresh).

Conclusion

With Concurrent Editing, the management of experimentation assets will be significantly easier. Experiment collaborators will be able to identify which pages are being concurrently edited to avoid overwriting and confusion. Concurrent editing will make cross-collaboration between users smoother by increasing experiment velocity and performance.

Questions?

Please contact support@optimizely.com if you need assistance with our Web Experimentation product. If you’d like to follow our development updates, feel free to subscribe to our RSS feed by clicking the corresponding icon below.

New: Optimizely Full Stack now offers starter kits for Fastly, Akamai, Cloudflare, Vercel, and AWS Lambda@Edge

Fri, 05 Aug 2022 15:23:46 GMT

By John Nguyen and Griffin Cox

In an increasingly competitive digital landscape where 100ms of added latency can cost 1% of sales, performance has never been more critical. All else equal, the fastest company wins.

Imagine you’re in California, using a simple messaging app. Let’s say you’re using an app powered by data centers on the other side of the world. You tap “view messages” and wait. And wait. Every millisecond you spend waiting is a waste of time. Your mind quickly wanders to something else. You may even start to consider finding a better app.

The time it takes to complete the roundtrip for a request to view your messages will be longer than if servers nearby would fulfill the request. The lag is much larger for intercontinental requests.

Recognizing this added time latency, companies in the 2000s began to operate multiple on-premises data centers that would route user requests to the nearest location. With the rapid growth in cloud computing, hosting and maintaining multiple data centers became easier, further improving performance. Content Delivery Networks (CDNs) emerged as a specialized way to serve data from the cloud as quickly and as cheaply as possible... that is until recent developments in Edge computing.

Companies looking to deliver content such as images, videos, and other assets blazingly fast are turning to ultra-distributed networks that seek to serve data as geographically close to the user as possible, sometimes as close as within the same city. These networks have come to be known as Edge networks. Numerous SaaS vendors such as Fastly, Akamai, Cloudflare, Vercel, and AWS Lambda@Edge are at the forefront of this technology.

Edge providers build highly scalable and localized content delivery networks that must operate with extraordinary resource constraints on memory, run time, and CPU usage to be financially feasible. Rather than run one server cluster in one data center, Edge networks host limited functionality at countless Edge locations as they are needed.

Here at Optimizely, we are committed to minimizing the performance impact of experimentation. At each stage of this technological arms race, we’ve supported the latest and greatest technologies so that our customers can save development effort and provide the best experience possible to their users. This means extending support for modern technologies as our customers adopt them. That’s why we’re proud to announce best-in-class support for Edge computing on five of the world’s most popular Edge networks: Fastly, Akamai, Cloudflare, Vercel, and AWS Lambda@Edge.

Edge Computing

Edge computing enables you to build fast and customizable experiences, incorporating the familiarity and convenience of serverless functions with the speed of close-proximity computation from the edges of a given network. You can think of it as analogous to a CDN wired up to serverless functions.

Edge computing provides the ideal mechanism for handling high-impact, low-cost decisions that directly influence user experiences – a perfect match for running experimentation using Optimizely.

Optimizely Edge Starter Kits

Today, using Edge computing services with Optimizely’s Full Stack SDK has been made easy with our brand-new edge starter kits available to help kickstart experimentation at the Edge, with support for five major platforms – Fastly, Akamai, Cloudflare, Vercel, and AWS Lambda@Edge.

Each starter kit provides an example implementation of the Optimizely Full Stack JavaScript Lite SDK that covers everything from importing and initializing the SDK and setting up user context to making decisions and returning the response of those decisions. In addition, each starter kit aligns to each platform’s specific space and time constraints and approaches to caching, data fetching, and response handling.

Using our Optimizely starter kits, you can spin up new edge computing instances ready for experimentation within minutes, all with the familiarity of JavaScript and our Full Stack SDKs.

You can view each starter kit’s developer documentation with step-by-step instructions here:

Benefits of using Optimizely @ Edge

Here are some benefits you'll find when using Optimizely at the edge, compared to our client-side and server-side solutions.

Balance Speed with Customization for User Experience

By computing, caching, and serving your data at the edge of the network, close to your users and/or source of data, edge computing provides all the speed of a CDN with the customizability of a serverless function. The starter kits enable you to utilize that speed and power for your use cases, from changing which assets are delivered based on decisions to making split-second decisions on user eligibility based on geographical location.

Reducing Overhead & Streamlining Developer Experience

Our edge starter kits provide an excellent developer experience, bringing the familiarity of JavaScript and our Full Stack SDKs into the world of edge computing. In addition, you’ll find that setting up an edge service may be easier than you think, as new services can be spun up in minutes and reduces the overhead of development compared to alternative methods of achieving the same balance of speed and customization.

Improved Security over Client-side Implementation

Making client-side decisions is fast, but insecure. Making decisions on your server and relaying them to your client is secure and slow. The middle ground is edge computing – providing both a fast and secure environment to serve decisions.

Ready to Get Started?

Check out our docs for step-by-step guides for each Edge provider. Our Customer Success Managers and Solutions Architects are here to help you, so don't hesitate to reach out to us with any questions you might have.

Easily Deploy Optimizely Agent to K8s via Helm Charts

Wed, 03 Aug 2022 16:57:49 GMT

By Mike Chu

Distributed systems pose new challenges to companies seeking to build experimentation platforms that support their business goals. In a monolith architecture that combines a frontend and backend, the implementation of one Optimizely Full Stack SDK may be sufficient to enable full stack experimentation. In headless architectures, where the frontend is decoupled from the backend, two SDKs may be sufficient. But, when the backend is broken down into microservices, organizations frequently turn to implementing one of our SDKs in each service, which can accrue high development and maintenance costs.

Microservice architectures have become the de facto standard for building cloud-native systems. Engineering organizations tasked with maintaining highly reliable, available, and scalable systems are also increasingly breaking down their monoliths into microservices. As high-performing organizations plug in additional capabilities, some orchestrate their ever-growing array of services with the help of Kubernetes to manage the workloads, provide scalability, and ultimately reduce their development, maintenance, and infrastructure costs.

The popularity of microservice architecture is why we introduced Optimizely Agent. With Agent companies can centralize their interactions with Optimizely in one place, eliminating the need to install SDKs in each microservice and reducing the maintenance costs over time as we update our platform. Until now, Agent was only available as a Docker image, with the expectation that our customers would need to deploy Agent optimally for their architectures with little guidance. Today, we’re happy to announce that it’s easier than ever to deploy Agent to Kubernetes (K8s) using Helm Charts.

Optimizely Agent

Agent is a microservice providing integration with Optimizely’s experimentation offering, allowing organizations to easily build and run experiments and deliver features across multiple environments and into existing applications without having to implement our language-specific SDKs in each service.

Optimizely Agent provides a RESTful API that exposes experimentation functionality as a service. Customers can run A/B tests, multivariate testing, and (soon) adaptive targeting experiments with much less implementation time and effort.

Ideal use cases for Agent

Incorporating Optimizely Agent into your tech stack can save you time and effort in the following situations:

Existing Service Oriented Architecture

If you already run a service-oriented architecture (SOA) Optimizely Agent can easily be included in your application stack to unlock your digital potential. SOA is a technique used to separate system functionality into discrete, but loosely coupled units. With Agent deployed to your infrastructure, your various microservices can then interact with Agent as a centralized point of decision evaluation and results reporting. The low implementation effort of Agent allows you to easily implement Optimizely experimentation across your existing operations.

Centralized Security & Privacy Compliance

Experiment decisions and event data flowing through your infrastructure are easier to identify when a specific service is involved. Securing and monitoring this traffic becomes much easier with Optimizely Agent.

For instance, consider your infrastructure administrators and infosec staff. They may need firewall rules configured for experimentation data on a containerized workload to comply with data handling regulations or company policies. By implementing Agent, you can easily configure these rules and regulations in one location.

Support for Multiple Programming Languages

Optimizely customers often leverage several of our software development kits (SDKs) depending on the product on which experiments are being run.

In some scenarios, especially in server-side applications, it can make sense to support a single implementation instead of coding the same solution in several programming languages.

This abstraction over HTTP using Optimizely Agent is instrumental if you have a nuanced tech stack or bespoke programming language in use. Leveraging Agent will bypass these hurdles when cultivating your culture of experimentation.

In cases where Optimizely does not offer an SDK for a chosen language, Agent can also stand in as a language-agnostic interface to Optimizely.

Future Proofing

Agent becomes a near plug-and-go component for your future initiatives. Running instances of Optimizely Agent as part of your infrastructure can provide current and future support for experimentation with both internal workloads and customer-facing experiences. As your company unlocks the tangible value of experimentation, you'll likely find other systems that can be optimized through testing.

How Is Optimizely Agent Typically Deployed?

Running via Docker

Optimizely Agent was designed to be executed in a container runtime, typically Docker. Using simple variables when starting the container, we offer customizability, such as networking options. We maintain two Agent images on Docker Hub, both based on the official Golang container image. The primary version of Agent uses Debian Linux, and we also support an Alpine Linux-based Agent image with a smaller footprint.

Engineers will often run a local instance on the Agent during development, but the Docker image can also be included as a part of a smaller swarm. Please review the Docker Hub page instructions to learn how to set up and run the Agent via Docker.

The Dockerized Optimizely Agent also supports the industry shift towards a declarative, infrastructure-as-code approach. Configuration via YAML files can easily be committed to your software repositories, versioned, and distributed in a controlled manner by leveraging orchestration technologies like Docker Compose, Ansible, Terraform, and especially Kubernetes.

Running From Source

It's rarely done, but the Golang-based Optimizely Agent can be built and run from the source code hosted on GitHub. Based on the Optimizely Go SDK, it gives Agent high-performance throughput.

Please review the README markdown instructions in the Agent repo for more information. We've included Windows, macOS, and Linux instructions and helper scripts.

We typically recommend that clients run Agent as a containerized service.

Running in Kubernetes

Kubernetes has become the leader in the container orchestration space. Running Agent in a multi-node cluster provides high availability and dovetails into an infrastructure-as-code paradigm.

As with many of the orchestration technologies mentioned earlier, there's complexity cost to consider. Developing and maintaining configuration files requires (in my best Liam Neeson voice) a very particular set of skills.

Fortunately, the DevOps community has a solution.

What is Helm?

Helm is a package manager that helps DevOps engineers develop, test, manage, publish, and distribute Kubernetes deployments. Helm reduces the effort of working with Kubernetes while adhering to container & infrastructure-as-code paradigms.

Helm helps Kubernetes engineers:

simplify sharing
manage complexity
easily update deployments
coordinate rollbacks (as needed)
perform repetitive operations consistently

Having developed my share of Kubernetes deployments, it's easy to find yourself copying and pasting chunks of YAML configurations to accomplish the above goals. Most developers will see this duplication and, after cringing, will look to satisfy patterns and reuse code, or in this case, develop templates.

To stick with Kubernetes' declarative approach, Helm has its own way of packaging K8s configurations for distribution.

What Are Helm Charts?

Helm Charts are a set of file manifests that combine YAML templating and basic programmatic flow to produce a native Kubernetes configuration. Helm handles generating and executing the output configuration against a target Kubernetes cluster. 

Developing Helm charts has a low barrier to entry and a high ROI.

Fortunately, in the context of the Optimizely Agent, we've done the work for you. We maintain a Helm Chart on Artifact Hub that can be used to install and manage an Agent deployment in your K8s cluster.

Minimize Development & Maintenance Effort for Agent via Helm Charts

We want to help you unlock the hidden value in your organization's digital properties. The faster we can incorporate experimentation into your technical operations, the sooner you can focus on designing experiments that return that value.

There are tangible advantages to deploying Optimizely Agent via Helm charts

Reduce the upfront effort needed to implement experimentation
Remove duplication of effort by centralizing decision support and reporting
Simplify deployment and upgrades to Agent
Codify desired values used by Agent

Infrastructure versioning (and rollbacks!)

Helm charts make it easy to scale your Agent deployment. Once you've integrated experimentation in your stack, you can easily add additional Agent instances to handle the increased load, as needed. When traffic decreases, you can remove extra instances, freeing resources on your cluster, and saving you money.

Templates and Values

Helm uses Go's template capabilities which provides a way to define how data is rendered into a Helm chart. This template and the values you choose merge to produce the necessary Kubernetes configuration files.

As a small but useful example, the fullNameOverride from the values.YAML file is used throughout the resulting K8s configuration YAML files. Keeping them synchronized is otherwise a manual effort.

Template Files

A Helm chart's templates directory contains all the templates used to generate the chart. Each Kubernetes resource needed by your cloud-native app stack, like Deployments, ConfigMaps, Secrets, Services, Persistent Volumes, etc., can have associated templates that Helm can fill in.

When you run helm install, Helm enumerates this templates directory and begins to load the declared values into each template file.

Values File

A values file allows you to define and set variables you can access in your templates. During installation, Helm uses the values you specify in your custom values.yaml file or via the --set flag in a command line interface (CLI) to override the defaults defined in the templates.

We recommend carefully reading the documentation for the Optimizely Agent Helm chart and the internal documentation for the values.yaml file.

Remember to commit your version of the Agent values.yaml to your version control system as a record of Optimizely Agent's configuration during each iteration of your implementation.

Common Optimizely Agent Helm Chart Values to Edit

The values you supply to your Optimizely Agent instance(s) will vary based on your infrastructure and experiment traffic. Here are some common variables to review from your custom values.YAML file supplied to the Helm chart:

replicaCount: Set an initial count of replicas based on an estimation of experiment traffic load

autoscaling: Configure how to scale Agent up and down and under which load situations
service: Use this section to configure how your Agent pods are expected to be available inside and outside your cluster
logs: Decide the log level and format of collected logs
config: Provide the configuration used in each instance of Agent

Remember that our Customer Success Managers are here to help you with the set-up and configuration of the Optimizely Agent, so don't hesitate to reach out to us with any questions you might have.

Integrate Optimizely with GA4

Thu, 23 Jun 2022 18:48:31 GMT

Google Universal Analytics and Google Analytics 4

In 2019, GA4 was introduced with the aim of helping businesses approach evolving measurement standards, which involved the following capabilities:

Measuring different kinds of data
Delivering a strong analytics experience
Visualizing unified user journeys across websites and apps
Predicting new insights with machine learning
Keeping up with a changing ecosystem (find out more here)

GA4 has become Google's recommended version of Google Analytics. Its previous versions – Universal Analytics/Classic Analytics – are some of the most popular analytics tools in use today, but GA4's updated capabilities have been pushing customers to migrate.

Here are some key differences:

	Universal Analytics	GA4
Measurement	Session-based	Flexible event-based
Reporting	Limited across platforms and devices	Fully enabled across platforms and devices
Debugging	No DebugView report available	DebugView report available
IP anonymization	Can be enabled/disabled	Enabled by default – can’t disable it due to privacy laws
Automation	Limited	Machine learning-based

On July 1, 2023, GA4 will officially replace Google Universal Analytics.

Integrate Optimizely with GA4

If you would like to migrate to GA4 now, we have several guides on how to integrate Optimizely Web or Full Stack, depending on your use case:

We will continue to provide further support as we get closer to the sunset of Universal Analytics in 2023.