How to Elevate Your Experimentation - Opticon workshop experience

As a non-expert in the field of experimentation, I’d like to share my feedback on the recent Opticon San Antonio workshop session titled "How to Elevate Your Experimentation Program."

I found the Optimizely session particularly valuable due to its focus on practical tools and actionable recommendations, which were presented as a framework to implement experimentation programs even without extensive experience in the field.

The experimentation exercise

The session featured a hands-on exercise where attendees were divided into groups of 4-6 people to work on a hypothetical company case study. The objective was to select and present experimentation program recommendations using an experimentation framework and a maturity worksheet containing common problem statements, underlying causes, and suggested actions.

During the exercise, each group analyzed the current experimentation program, business priorities, team collaboration dynamics, active channels, challenges, and goals. We then aligned this information with the maturity pillars and problems outlined in the recommendations table.

Initially, the exercise felt overwhelming because the company overview information was presented in a raw and somewhat disorganized manner, making it difficult to identify the company’s needs. However, after a few iterative cycles of reviewing the case study and consulting the experimentation recommendations worksheet, we began to recognize problem statements that matched the company’s issues and goals. Ultimately, we were able to select and present several well-founded experimentation recommendations. These were grounded in the advanced framework and supported by data rather than relying solely on “common sense,” “innovation,” or “prior experience.”

This initial sense of being overwhelmed mirrored the challenges faced when analyzing real-world data, which is often messy and unclear. Without a structured maturity worksheet and a foundation of experimentation problems and recommendations, it’s easy to feel lost or confused by irrelevant or misleading information.

Conclusion

I consider this exercise and the Optimizely experimentation framework immensely valuable for companies with limited experience in experimentation or those aiming to enhance their experimentation programs. It minimizes the risk of failure and avoids wasting time and money on solutions that don’t address their needs. Moreover, it serves as an excellent starting point for identifying hidden issues or opportunities within their systems.

Thanks Vimi Kaul and Sama Asali from Optimizely for this great and useful experience.

Resources:

Case study

Matury Worksheet