<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom"><title type="text">Blog posts by Polly Walton</title><link href="https://world.optimizely.com/blogs/polly-walton/" /><updated>2026-11-12T10:16:22.0000000Z</updated><id>https://world.optimizely.com/blogs/polly-walton/</id><generator uri="https://world.optimizely.com/blogs/polly-walton/" version="2.0">Optimizely World</generator><entry><title>Your weekly experiment report is one slash (/) away with Mark</title><link href="https://world.optimizely.com/blogs/polly-walton/dates/2026/11/your-weekly-experiment-report-is-one-slash-away/" /><id>https://world.optimizely.com/blogs/polly-walton/dates/2026/11/your-weekly-experiment-report-is-one-slash-away/</id><published>2026-11-12T10:16:22.0000000Z</published><updated>2026-11-12T10:16:22.0000000Z</updated><author><name>polly.walton@niteco.com</name></author><category term="Blog post" /><summary type="html">&lt;p&gt;&lt;em&gt;How to build a reusable report skill in Mark for Optimizely Web Experimentation.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;I want to share something with my fellow CROs. I love using Mark with Optimizely Web Experimentation because it makes those time-consuming tasks much easier. You know the ones: monotonous, boring, and they drain your week.&lt;/p&gt;
&lt;div&gt;I do not need to write any code. I just describe what I want in a skill, and that&amp;rsquo;s it. Creating skills is very easy.&lt;/div&gt;
&lt;p&gt;Weekly reporting is a pain for most experimentation teams. We all know it. Here&amp;rsquo;s how I made it easier.&lt;/p&gt;
&lt;div&gt;Every week, someone asks how the test is going. Then someone copies numbers into a slide and writes the same three sentences again.&lt;/div&gt;
&lt;div&gt;&amp;nbsp;&lt;/div&gt;
&lt;div&gt;I solved this with a skill in Mark. I type&lt;span style="background-color: rgb(206, 212, 217);"&gt; /weekly-experiment-report&lt;/span&gt; and an experiment ID, and Mark gives me a clean, one-page PDF. This post will show you how to build the same skill. The full skill file is ready to download. Feel free to copy it and make it your own, courtesy of me, Polly from Niteco.&lt;/div&gt;
&lt;h2&gt;At a glance&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;What it does:&lt;/strong&gt; Turns one experiment ID into a one-page A4 report.&lt;br /&gt;&lt;strong&gt;Who it is for:&lt;/strong&gt; Brands that run tests in Optimizely Web Experimentation.&lt;br /&gt;&lt;strong&gt;What you need:&lt;/strong&gt; A Mark account where you can add skills, and an experiment ID.&lt;br /&gt;&lt;strong&gt;Time:&lt;/strong&gt; A few minutes to set up. Seconds to run.&lt;/p&gt;
&lt;h2&gt;What you get&lt;/h2&gt;
&lt;p&gt;The report has a section for each brand. Each section shows control versus variant, the lift for every metric, and a short analysis. At the bottom, there's a box with the next steps.&lt;/p&gt;
&lt;p&gt;My favourite rule is the colour code. A lift only turns green or red when it reaches 90% confidence. Anything below that stays grey. No more celebrating a 3% lift on a tiny sample.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/b6580af893a141518482eb8342982c6b.aspx" alt="One page A4 experiment report with a results table for each brand, an analysis box and a Next steps block." width="500" height="434" /&gt;&lt;br /&gt;&lt;span style="color: rgb(126, 140, 141);"&gt;&lt;em&gt;Image: One-page A4 experiment report with a results table for each brand, an analysis box and a Next steps block.&lt;/em&gt;&lt;/span&gt;&lt;/p&gt;
&lt;h2&gt;Before you start&lt;/h2&gt;
&lt;ol&gt;
&lt;li&gt;Copy the skill file text below.&lt;/li&gt;
&lt;li&gt;Find every XXXX. Replace each one with your own brand name, short name and Optimizely project ID. The XXXX is loud on purpose. It is hard to miss, so nobody sends a client a report that says XXXX.&lt;/li&gt;
&lt;li&gt;Check your access. Organisation skills need the right permission. No access yet? Choose Personal when you build the skill, then ask your admin to share it later.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The last step of the skill uses two Mark tools, &lt;span style="background-color: rgb(206, 212, 217);"&gt;write_content_to_file&lt;/span&gt; and &lt;span style="background-color: rgb(206, 212, 217);"&gt;convert_to_pdf&lt;/span&gt;, to save the report and turn it into a PDF. If Mark cannot find them, ask your admin to switch them on.&lt;/p&gt;
&lt;h2&gt;Build it in Mark&lt;/h2&gt;
&lt;p&gt;The skill form has a handful of fields. Here is what I put in each one.&lt;/p&gt;
&lt;p&gt;Prefer to chat? Ask Mark to "Help me create a skill" and it fills in the form for you. Then check each field against the steps below.&lt;/p&gt;
&lt;h3&gt;1. Open Skills and add a new skill&lt;/h3&gt;
&lt;p&gt;In Mark, go to &lt;strong&gt;Context&lt;/strong&gt; and choose &lt;strong&gt;Create new skill&lt;/strong&gt;.&lt;br /&gt;&lt;br /&gt;&lt;img src="/link/2b4114c8f6e341739f4f08ef0c88dcfe.aspx" alt="" width="400" height="412" /&gt;&lt;/p&gt;
&lt;p&gt;then&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/d691c4e8a380403cb82093ece7e0eff1.aspx" alt="" width="300" height="152" /&gt;&lt;/p&gt;
&lt;h3&gt;2. Name your skill&lt;/h3&gt;
&lt;p&gt;Type &lt;strong&gt;Weekly Experiment Report&lt;/strong&gt; in the Name field. Keep it plain. Your whole team will see it in a list.&lt;/p&gt;
&lt;h3&gt;3. Add a shortcut&lt;/h3&gt;
&lt;p&gt;Type &lt;strong&gt;weekly-experiment-report&lt;/strong&gt; in the Shortcut field. Mark shows the slash version under the box. A green Available means nobody has taken it. This is what you will type in chat.&lt;/p&gt;
&lt;p&gt;Keep the Active switch on, at the top right, so Mark can use the skill.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/6c5aef2242d544b8bdb9b37b6240ada4.aspx" alt="" width="600" height="261" /&gt;&lt;/p&gt;
&lt;h3&gt;4. Choose who it is for&lt;/h3&gt;
&lt;p&gt;Pick Organisation to share it with your team. Pick Personal to keep it to yourself while you test.&lt;/p&gt;
&lt;h3&gt;5. Say when to use it&lt;/h3&gt;
&lt;p&gt;Keep Keywords or phrases selected under When to use. Then add one sentence about the job. Here is mine.&lt;/p&gt;
&lt;p style="padding-left: 20pt;"&gt;&lt;em&gt;&lt;strong&gt;Give it an Optimizely Web Experimentation experiment ID and get a weekly results report as a one-pager PDF.&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Mark uses this to decide when your skill fits. Write it the way your team would ask.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/b34cec98a5cd4ff3865681c61b7b7df5.aspx" alt="" width="500" height="114" /&gt;&lt;/p&gt;
&lt;h3&gt;6. Paste the core skill&lt;/h3&gt;
&lt;p&gt;Copy the skill text from your download or from the end of this post. Paste it into Core Skill. Start at the line that begins "Turns an Optimizely Web Experimentation experiment ID". Leave out the title and anything above it, because the Name field already does that job.&lt;/p&gt;
&lt;p&gt;At the top of the skill you will see the XXXX table. Fill it in now.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/b083020c87e34e539a5d314c516d7cc2.aspx" alt="" width="500" height="190" /&gt;&lt;/p&gt;
&lt;h3&gt;7. Skip tags and files&lt;/h3&gt;
&lt;p&gt;Tags and Files are optional. I left both empty.&lt;/p&gt;
&lt;h3&gt;8. Pick your product&lt;/h3&gt;
&lt;p&gt;Scroll down to Where to use. Choose &lt;strong&gt;Experimentation&lt;/strong&gt; in the Product drop-down. Choose &lt;strong&gt;All &lt;/strong&gt;in Instance to use the skill on every instance, or pick one. Need another product? Use the plus button to add a row.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/45e7982471dc486fbb655c4345277992.aspx" alt="" width="500" height="55" /&gt;&lt;/p&gt;
&lt;h3&gt;9. Save&lt;/h3&gt;
&lt;p&gt;Click Save. Your skill is ready to use.&lt;/p&gt;
&lt;h2&gt;Try it&lt;/h2&gt;
&lt;p&gt;Open a new chat with Mark. In the Ask Mark box, type / and pick Weekly Experiment Report from the list. You can also type &lt;span style="background-color: rgb(206, 212, 217);"&gt;/weekly-experiment-report&lt;/span&gt;. Then give Mark your &lt;strong&gt;experiment ID&lt;/strong&gt;. Forgot it? Mark will ask.&lt;/p&gt;
&lt;p&gt;Mark finds the experiment results directly from the Optimizely Web Experimentation tool, builds the report and gives you a PDF link.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/1096a7d87fa64932b38d4e9959c5721a.aspx" alt="" width="500" height="143" /&gt;&lt;/p&gt;
&lt;p&gt;Check the numbers against Optimizely the first time you run it. The skill tells Mark never to make up a number. If a value is missing, the report shows n/a. A quick check still pays off.&lt;/p&gt;
&lt;h2&gt;Make it yours&lt;/h2&gt;
&lt;p&gt;Everything in the skill is plain text, so you can change it.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Brands&lt;/strong&gt;. Swap the XXXX rows for your own. Only one project? Delete a row.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Colours&lt;/strong&gt;. My report uses Niteco inspired colours. The skill lists every hex code, so change them to match your brand.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Rules&lt;/strong&gt;. The 90% confidence rule lives in the skill. Want 95%? Change the number.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Metrics&lt;/strong&gt;. Add the metrics your team cares about. The tables take up to six.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Change one thing at a time and run the skill again. That way you know what made the difference.&lt;/p&gt;
&lt;h2&gt;The full skill&lt;/h2&gt;
&lt;p&gt;Here is the core skill, ready to copy. It starts at the line you need for the Core Skill box, so you can paste it as it is. Remember to replace every XXXX first.&lt;/p&gt;
&lt;h2&gt;&lt;span style="color: rgb(224, 62, 45);"&gt;-- !!copy and paste this!! --&lt;/span&gt;&lt;/h2&gt;
&lt;p&gt;Turns an Optimizely Web Experimentation experiment ID into a one-page A4 report that anyone can read in 30 seconds.&lt;/p&gt;
&lt;h2&gt;Before you use this skill&lt;/h2&gt;
&lt;p&gt;Replace every XXXX in this skill with your own details. Do this once, then save the skill.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Brand&lt;/th&gt;
&lt;th&gt;Short name&lt;/th&gt;
&lt;th&gt;Optimizely project ID&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;XXXX&lt;/td&gt;
&lt;td&gt;XXXX&lt;/td&gt;
&lt;td&gt;XXXXXXXXXX&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;XXXX&lt;/td&gt;
&lt;td&gt;XXXX&lt;/td&gt;
&lt;td&gt;XXXXXXXXXX&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Only have one project? Delete the second row. Have more? Add rows.&lt;/p&gt;
&lt;h2&gt;Steps&lt;/h2&gt;
&lt;h3&gt;1. Gather inputs&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;Ask for the experiment ID or test name if the user has not given it.&lt;/li&gt;
&lt;li&gt;If anything is unclear, ask before you start. This includes metrics, segments, the snapshot date, or whether to use another source such as GA4 instead of Optimizely.&lt;/li&gt;
&lt;li&gt;Ask one thing at a time.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;2. Get the experiment data&lt;/h3&gt;
&lt;p&gt;Look for the experiment in each project in the table above. Use whatever access you have: the Optimizely Experimentation API, a connector, or an export.&lt;/p&gt;
&lt;p&gt;Pull this for each brand:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Total visitors&lt;/li&gt;
&lt;li&gt;Days running. Count active days only and leave out any paused periods.&lt;/li&gt;
&lt;li&gt;Control rate for each metric&lt;/li&gt;
&lt;li&gt;Variant lift and confidence level (CL) for each metric, per segment (All users, Desktop, Mobile)&lt;/li&gt;
&lt;li&gt;Clicks and click rate for any banner or button metrics&lt;/li&gt;
&lt;li&gt;Test status&lt;/li&gt;
&lt;li&gt;Snapshot date, which is the day the results were pulled&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Rules:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Never invent or estimate a number. If a value is missing, show "n/a" and tell the user.&lt;/li&gt;
&lt;li&gt;If the experiment sits in one project only, make one brand block.&lt;/li&gt;
&lt;li&gt;If the test targets one device only, show one control row and one variant row. Skip the other segments.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;3. Build the HTML report&lt;/h3&gt;
&lt;p&gt;Make one A4 portrait page with Tailwind CSS and the Inter font.&lt;/p&gt;
&lt;p&gt;Page: &lt;code&gt;width: 210mm; min-height: 297mm; padding: 10mm 12mm&lt;/code&gt;. Use a flex column so the Next Steps block sits at the bottom of the page.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Header&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Small grey label: "Optimizely Experimentation Performance Report", then the environment and experiment ID.&lt;/li&gt;
&lt;li&gt;Test name in bold capitals.&lt;/li&gt;
&lt;li&gt;Goal in grey below the name, in one sentence.&lt;/li&gt;
&lt;li&gt;Snapshot date on the right.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Colours&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The report uses Niteco inspired colours. Swap them for your own brand colours if you like. In Tailwind, use arbitrary values such as &lt;code&gt;bg-[#F0F0E6]&lt;/code&gt;.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Use&lt;/th&gt;
&lt;th&gt;Colour&lt;/th&gt;
&lt;th&gt;Hex&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Page background&lt;/td&gt;
&lt;td&gt;White&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#FFFFFF&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Main text&lt;/td&gt;
&lt;td&gt;Niteco black&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#0F0F14&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Muted text&lt;/td&gt;
&lt;td&gt;Grey&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#6F6F72&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Accent (brand bar, highlights)&lt;/td&gt;
&lt;td&gt;Niteco yellow&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#FFE628&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Soft panels and highlight rows&lt;/td&gt;
&lt;td&gt;Warm grey&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#F0F0E6&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Borders&lt;/td&gt;
&lt;td&gt;Warm grey&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#E6E6DC&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Good news&lt;/td&gt;
&lt;td&gt;Signal green&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#19A550&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Bad news&lt;/td&gt;
&lt;td&gt;Red (outside the Niteco palette, kept for clarity)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#E11D48&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Use one accent colour at a time and keep the page mostly white.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Status badge&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;A rounded badge at the top right. It shows the overall status:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Test Running: light green fill &lt;code&gt;#B9FAC8&lt;/code&gt;, dark text &lt;code&gt;#0F0F14&lt;/code&gt;, green dot &lt;code&gt;#19A550&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Test Paused: light yellow fill &lt;code&gt;#FFF8BE&lt;/code&gt;, dark text &lt;code&gt;#0F0F14&lt;/code&gt;, yellow dot &lt;code&gt;#FFE628&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Test Ended: warm grey fill &lt;code&gt;#E6E6DC&lt;/code&gt;, dark text &lt;code&gt;#0F0F14&lt;/code&gt;, grey dot &lt;code&gt;#6F6F72&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Brand blocks&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Give each brand its own block.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Vertical yellow bar (&lt;code&gt;#FFE628&lt;/code&gt;) on the left of the brand header.&lt;/li&gt;
&lt;li&gt;Brand name in bold capitals.&lt;/li&gt;
&lt;li&gt;Small status badge next to the name, in the same colours as the status badge above: Running, Resumed or Variant Won (green), Paused (yellow) or Ended (grey).&lt;/li&gt;
&lt;li&gt;Right-aligned traffic details: "24 DAYS RUNNING &amp;bull; 78,841 VISITORS".&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Tables&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;One column per metric. Use up to six.&lt;/li&gt;
&lt;li&gt;Balance the column widths so no header wraps onto more than two lines. Watch long names like "Successful Account Creation".&lt;/li&gt;
&lt;li&gt;"All users" row: warm grey background (&lt;code&gt;#F0F0E6&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;Control row: grey italic text (&lt;code&gt;#6F6F72&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;Winning variant row: soft green background (&lt;code&gt;#B9FAC8&lt;/code&gt; at about 45% strength).&lt;/li&gt;
&lt;li&gt;Under each lift, show the confidence level in small grey text, for example "&amp;lt;10% CL".&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Lift colours&lt;/strong&gt;&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Confidence level&lt;/th&gt;
&lt;th&gt;Positive lift&lt;/th&gt;
&lt;th&gt;Negative lift&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;90% or higher&lt;/td&gt;
&lt;td&gt;Green (&lt;code&gt;#19A550&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;Red (&lt;code&gt;#E11D48&lt;/code&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Below 90%&lt;/td&gt;
&lt;td&gt;Dark grey (&lt;code&gt;#3F3F43&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;Dark grey (&lt;code&gt;#3F3F43&lt;/code&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Below 90% CL, always use neutral grey, even when the lift looks big.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Analysis box&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Warm off-white background (&lt;code&gt;#FAFAF0&lt;/code&gt;) with a thin &lt;code&gt;#E6E6DC&lt;/code&gt; border.&lt;/li&gt;
&lt;li&gt;Bold "Analysis" title with a bottom border line. Show the sample size on the right of the title.&lt;/li&gt;
&lt;li&gt;Write 3 or 4 short sentences. Start with the result, then give the sample size, then the main risk.&lt;/li&gt;
&lt;li&gt;Below 90% CL, call a result "directional". Do not say "won" or "significant".&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Next steps block&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Niteco black background (&lt;code&gt;#0F0F14&lt;/code&gt;), white text, yellow heading (&lt;code&gt;#FFE628&lt;/code&gt;) and yellow bullet dots.&lt;/li&gt;
&lt;li&gt;Two or three bullets. Each one starts with a bold action, then one sentence on why.&lt;/li&gt;
&lt;li&gt;Choose from actions like: Continue running, Conclude in X weeks, Roll out the winner, Pause, Re-test with changes, Roll out on one device only.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;4. Check before you save&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;The report fits on one page. Nothing is cut off, including every Next Steps bullet.&lt;/li&gt;
&lt;li&gt;Every number matches the source data.&lt;/li&gt;
&lt;li&gt;Lift colours follow the 90% rule.&lt;/li&gt;
&lt;li&gt;No placeholder text is left in the report.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;5. Save and convert to PDF&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;Save the compiled HTML file with the &lt;code&gt;write_content_to_file&lt;/code&gt; tool. Use a clean file name, for example &lt;code&gt;A4_Report_[Test_Name].html&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Pass the file download link to the &lt;code&gt;convert_to_pdf&lt;/code&gt; tool.&lt;/li&gt;
&lt;li&gt;Share the PDF link with the user.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;6. Reply to the user&lt;/h3&gt;
&lt;p&gt;Share the PDF link and add two or three lines on the result. Tell the user about any missing data or numbers you could not check.&lt;/p&gt;
&lt;h2&gt;&lt;span style="color: rgb(224, 62, 45);"&gt;-- !!end of copy and paste!! --&lt;/span&gt;&lt;/h2&gt;
&lt;p&gt;&amp;nbsp;&lt;/p&gt;
&lt;h2&gt;Final thoughts&lt;/h2&gt;
&lt;p&gt;A skill like this will not replace your judgement. It takes away the copy and paste, so you can spend your time on what the numbers mean.&lt;/p&gt;
&lt;p&gt;Try it, then tell me what you would add. A new metric? A different colour rule? I would love to see your versions.&lt;/p&gt;
&lt;p&gt;Happy testing and reporting,&lt;/p&gt;
&lt;p&gt;Polly&lt;/p&gt;
&lt;p&gt;&amp;nbsp;&lt;/p&gt;
&lt;h2&gt;How can Niteco help you?&lt;/h2&gt;
&lt;p&gt;Niteco is the largest global Optimizely partner and Optimizely AI Innovator of the Year and Mark AI Preferred Partner. Our vast team knows CMS, Commerce, CMP, ODP, Experimentation and the Optimizely Agentic Platform inside and out.&lt;/p&gt;
&lt;table&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;th&gt;
&lt;p&gt;Number&lt;/p&gt;
&lt;/th&gt;
&lt;th&gt;
&lt;p&gt;What it means&lt;/p&gt;
&lt;/th&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td colspan="1"&gt;
&lt;p&gt;&lt;strong&gt;#1&lt;/strong&gt;&lt;/p&gt;
&lt;/td&gt;
&lt;td colspan="1"&gt;
&lt;p&gt;Optimizely's largest global partner&lt;/p&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td colspan="1"&gt;
&lt;p&gt;&lt;strong&gt;400+&lt;/strong&gt;&lt;/p&gt;
&lt;/td&gt;
&lt;td colspan="1"&gt;
&lt;p&gt;AI-trained experts, certified across Optimizely, Microsoft, Contentful and Adobe&lt;/p&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td colspan="1"&gt;
&lt;p&gt;&lt;strong&gt;2000+&lt;/strong&gt;&lt;/p&gt;
&lt;/td&gt;
&lt;td colspan="1"&gt;
&lt;p&gt;Successful projects delivered&lt;/p&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td colspan="1"&gt;
&lt;p&gt;&lt;strong&gt;9&lt;/strong&gt;&lt;/p&gt;
&lt;/td&gt;
&lt;td colspan="1"&gt;
&lt;p&gt;Optimizely MVPs, the largest team of them&lt;/p&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Want help with your upgrade, migration, commerce or experimentation plans? Or want to get more out of Mark and the Optimizely Agent Platform? &lt;a href="https://niteco.com/contact-us/"&gt;Get in touch with Niteco&lt;/a&gt; and let's talk.&lt;/p&gt;
&lt;p&gt;&amp;nbsp;&lt;/p&gt;</summary></entry><entry><title>We Cloned Our Best Analyst with AI: How Our Opal Hackathon Grand Prize Winner is Changing Experimentation</title><link href="https://world.optimizely.com/blogs/polly-walton/dates/2026/2/we-cloned-our-best-analyst-with-ai-how-our-opal-hackathon-grand-prize-winner-is-changing-experimentation/" /><id>https://world.optimizely.com/blogs/polly-walton/dates/2026/2/we-cloned-our-best-analyst-with-ai-how-our-opal-hackathon-grand-prize-winner-is-changing-experimentation/</id><published>2026-02-16T15:41:22.0000000Z</published><updated>2026-02-16T15:41:22.0000000Z</updated><author><name>polly.walton@niteco.com</name></author><category term="Blog post" /><summary type="html">&lt;p&gt;Every experimentation team knows the feeling. You have a backlog of experiment ideas, but progress is bottlenecked by one critical team member, the Analyst.&lt;/p&gt;
&lt;p&gt;Our analyst is the hero of our experimentation programmes. She&amp;rsquo;s the one who dives into Google Analytics, digs for data and runs the complex calculations for every single test idea. Sample size, baseline metrics, and test duration. It all flows through her. But she&amp;rsquo;s only one person. The process is slow, repetitive and a classic bottleneck. I can&amp;rsquo;t count the number of times I&amp;rsquo;ve jokingly said to her:&lt;/p&gt;
&lt;h3&gt;&lt;em&gt;&lt;strong&gt;&amp;ldquo;I wish I could clone you!&amp;rdquo;&lt;/strong&gt;&lt;/em&gt;&lt;/h3&gt;
&lt;p&gt;That offhand comment sparked an idea. During a brainstorming session with &lt;a href="https://niteco.com/"&gt;Niteco&lt;/a&gt;'s AI R&amp;amp;D department for the Optimizely Opal Hackathon in New York, we asked ourselves: what would be the biggest-impact solution we could build with AI? I immediately thought: what if we could clone our analyst? What if we could build an AI version of her, an &amp;ldquo;Experimentation Strategist AI Worker&amp;rdquo; that could handle the soul-crushing, repetitive work, freeing us up to focus on strategy and insights?&lt;/p&gt;
&lt;p&gt;That idea didn&amp;rsquo;t just get us excited. It won us the &lt;strong&gt;Grand Prize&lt;/strong&gt; at the Opal Hackathon in New York.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/c2675a2804a148e3ad23ca808e964a5b.aspx" alt="" width="545" height="333" /&gt;&lt;/p&gt;
&lt;p&gt;I want to take you behind the scenes of our award-winning solution and show you how we&amp;rsquo;re using Optimizely Opal to build an AI teammate for every experimentation team.&lt;/p&gt;
&lt;h2&gt;Meet your new AI teammate: The Experimentation Strategist AI Worker&lt;/h2&gt;
&lt;p&gt;At its heart, our solution (which we affectionately call the &amp;ldquo;Sample Size Genie&amp;rdquo;) is a specialised AI Worker built natively on Optimizely Opal. It&amp;rsquo;s not a standalone tool or a fancy chatbot; it&amp;rsquo;s a fully orchestrated, multi-agent workflow that automates the entire experiment planning process.&lt;/p&gt;
&lt;p&gt;It&amp;rsquo;s designed for anyone on the team, regardless of their technical skill. You simply talk to it in plain English.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You can say:&lt;/strong&gt; &lt;em&gt;"Calculate the sample size for a test on our product detail pages, focusing on desktop traffic in the US market."&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/c67115a41ba641a18f74e8cd459597b6.aspx?1771256314210" width="330" /&gt;&lt;/p&gt;
&lt;p&gt;Instantly, our AI Worker gets to work. It connects to your GA4 property or multiple properties, identifies all the pages that match the &amp;ldquo;product detail page&amp;rdquo; pattern, pulls the relevant traffic and baseline conversion metrics, applies the segmentation you asked for, and comes back in seconds with the sample size, estimated test duration and a recommendation on the test&amp;rsquo;s feasibility.&lt;br /&gt;&lt;br /&gt;&lt;img src="/link/62af4c0571514f5c83068d28dc26dfb4.aspx?1771256376246" width="330" /&gt;&lt;/p&gt;
&lt;p&gt;No more spreadsheets. No more waiting for days. Just answers.&lt;/p&gt;
&lt;h2&gt;How it works (without getting too technical)&lt;/h2&gt;
&lt;p&gt;The magic lies in how we used Opal&amp;rsquo;s native architecture. We didn&amp;rsquo;t just write a clever prompt; we built a collaborative team of two specialised AI agents that work together seamlessly.&lt;/p&gt;
&lt;h3&gt;1. The Data Acquisition Agent&lt;/h3&gt;
&lt;p&gt;This agent is the researcher. Its job is to understand your request, connect to the right data source (such as GA4), retrieve the exact metrics you need and structure the information. If it can&amp;rsquo;t connect to GA4 or if you&amp;rsquo;re testing a brand-new feature with no historical data, it&amp;rsquo;s smart enough to fall back on industry benchmarks (such as Dynamic Yields published benchmarks) or even ask you for manual input.&lt;/p&gt;
&lt;h3&gt;2. The Analysis &amp;amp; Recommendation Agent&lt;/h3&gt;
&lt;p&gt;This agent is the strategist. It takes the structured data from the first agent and gets to work. It applies the correct statistical formulas, calculates the sample size and test duration and most importantly, assesses the viability of the experiment. If the traffic is too low to yield meaningful results, it won&amp;rsquo;t just say "no". It will recommend alternatives, like expanding the target audience or adjusting the expected uplift.&lt;/p&gt;
&lt;p&gt;These two agents collaborate through &lt;strong&gt;Opal&amp;rsquo;s Orchestration Layer&lt;/strong&gt;, passing data and context back and forth and using a shared Tool Registry to invoke specific functions, such as a calculator or the GA4 Data API. It&amp;rsquo;s a true AI team, working in perfect sync.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Technical diagram&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/745cfc869e114961ae3dc4917bfd19f7.aspx" width="330" height="332" /&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;br /&gt;Workflow details&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/2da433bc0c0843cd891ac6ffc9ce2eba.aspx?1771255480832" width="330" /&gt;&lt;/p&gt;
&lt;h2&gt;Solving FOUR problems in one go&lt;/h2&gt;
&lt;p&gt;What impressed the judges at the hackathon was how this single solution solves four common experimentation headaches:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Direct GA4 integration: get sample sizes based on real, live data in seconds.&lt;/li&gt;
&lt;li&gt;Limited traffic segments: receive specialist recommendations when testing on niche audiences.&lt;/li&gt;
&lt;li&gt;Industry benchmarks: plan tests for new features with confidence, even without historical data.&lt;/li&gt;
&lt;li&gt;Manual fallback: keep moving forward even if a data connection fails.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2&gt;It is a game-changer&lt;/h2&gt;
&lt;p&gt;This isn't about replacing analysts. It&amp;rsquo;s about augmenting them. By automating the 80% of their job that is repetitive calculation, we free them up to focus on the 20% that requires human touch: imagination, deep analysis, strategic thinking and nurturing a culture of experimentation.&lt;/p&gt;
&lt;p&gt;We're already building the next evolution of this AI Worker - one that can create many test ideas, generate multiple design variations directly from Figma AI, hold a dialogue with users just like they would with a UX designer about variant designs, and then build the variant code and insert it directly in Optimizely Web Experimentation. Imagine going from nothing to a fully built experimentation roadmap with no analyst, no designer or developer &lt;strong&gt;bottleneck&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;That&amp;rsquo;s the future of experimentation: a world where Opal empowers our creativity and every team has an AI teammate to help them move faster, test smarter, and deliver better results.&lt;/p&gt;
&lt;p&gt;We're planning to give away this and other AI Workers for free, even to companies that aren't&amp;nbsp;&lt;a href="https://niteco.com/"&gt;Niteco&lt;/a&gt;'s clients. This is so everyone in the Optimizely ecosystem can benefit and grow. As they say, &amp;ldquo;A rising tide lifts all boats&amp;rdquo;. By we, I mean the OMVPs at Niteco and by &amp;lsquo;free&amp;rsquo; I mean the solution itself will be free, but of course, to use it, you will need to have your own Opal credits. I just want to clarify it. &lt;br /&gt;&lt;br /&gt;If you are interested in finding out more, reach out to me on LinkedIn.&amp;nbsp;&lt;/p&gt;</summary></entry><entry><title>Experimentation Evolution with AI (Masterclass Recap)</title><link href="https://world.optimizely.com/blogs/polly-walton/dates/2025/11/test/" /><id>https://world.optimizely.com/blogs/polly-walton/dates/2025/11/test/</id><published>2025-11-21T04:45:01.0000000Z</published><updated>2025-11-21T04:45:01.0000000Z</updated><author><name>polly.walton@niteco.com</name></author><category term="Blog post" /><summary type="html">&lt;p&gt;If you think you are not using AI in your experimentation program you are probably wrong. Ever asked an AI to rephrase a hypothesis or brainstorm a few ideas? That is a start. But you are just scratching the surface.&lt;/p&gt;
&lt;p&gt;The real opportunity is not in automating small tasks. It is in transforming your entire workflow. In a recent Optimizely Academy masterclass I have explained how AI is making experimentation faster smarter and more accessible for everyone.&lt;/p&gt;
&lt;p&gt;This article recaps the key insights from my session. It is a practical guide to upgrading your experimentation program with AI to get better results with less effort.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://academy.optimizely.com/student/page/2820737-masterclass-experimentation-evolution-with-ai"&gt;Experimentation Evolution wiht AI - Optimizely Academy Masterclass&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/0587e52d94bb4238834f1d8fc1393f02.aspx" alt="" width="600" height="299" /&gt;&lt;/p&gt;
&lt;h2&gt;The old way is broken&lt;/h2&gt;
&lt;p&gt;Let&amp;rsquo;s be honest. Traditional experimentation is hard. It is slow expensive and often requires a team of specialists. This creates bottlenecks that kill momentum and cause stakeholders to lose interest. It is a story many of us know all too well.&lt;/p&gt;
&lt;p&gt;But it does not have to be this way.&lt;/p&gt;
&lt;h2&gt;AI is changing the game&lt;/h2&gt;
&lt;p&gt;AI is not a futuristic dream. It is here now and it is changing everything. What used to take days or weeks can now be done in minutes. We are compressing years of effort into weeks.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/948eca40157c432baceb8ab71fb20536.aspx" alt="" width="600" height="323" /&gt;&lt;/p&gt;
&lt;p&gt;AI can help with every step of the process from generating ideas to analyzing results. Here are five areas where AI is making a huge impact.&lt;/p&gt;
&lt;h3&gt;1. Idea generation on steroids&lt;/h3&gt;
&lt;p&gt;A great experimentation program needs a constant flow of fresh ideas. AI can be your tireless brainstorming partner. Tools like Optimizely Opal and ChatGPT can generate an endless stream of testable hypotheses. The key is to be specific. Give the AI context data and clear objectives.&lt;/p&gt;
&lt;h3&gt;2. Calculations without the headache&lt;/h3&gt;
&lt;p&gt;Calculating sample sizes and test durations is a massive bottleneck. It is a tedious and error-prone process that can stall your program before it even starts. At Niteco we built an AI agent called the Sample Size Genie to solve this problem. It connects to your GA4 data and provides accurate calculations in seconds. It even offers strategic advice to help you design better experiments. This tool recently won the grand prize at Optimizely&amp;rsquo;s hackathon in New York.&lt;/p&gt;
&lt;p&gt;In the masterclass I showed how I went from a vague idea to a solid experiment plan in under a minute. That is a task that would have taken a specialist analyst an hour to do manually.&lt;/p&gt;
&lt;p&gt;I did a demo of how Sample Size Genie works - check it out! All you need is Opal credits.&amp;nbsp;&lt;/p&gt;
&lt;h3&gt;3. Design without the delay&lt;/h3&gt;
&lt;p&gt;Design is another common bottleneck. But with AI-powered tools like Figma Make you can create professional designs in minutes. Just describe what you want and the AI does the rest. This frees up your designers to focus on more strategic work while you get the high-quality variations you need to test.&lt;/p&gt;
&lt;h3&gt;4. Development without the drudgery&lt;/h3&gt;
&lt;p&gt;For years code has been the biggest bottleneck of all. But AI is automating the coding process. In the masterclass I have demonstrated how to build a new experiment variant in minutes using an AI tool called v0. This is a game-changer. This functionality is going to be native in Optimizely CMS and Web Experimentation in 2026. The brands that are using AI to eliminate these bottlenecks are running 10 to 20 times more tests with the same resources.&lt;/p&gt;
&lt;h3&gt;5. Results analysis on demand&lt;/h3&gt;
&lt;p&gt;Analyzing results can be complex. Optimizely&amp;rsquo;s &amp;ldquo;Summarize Results&amp;rdquo; feature is a great starting point. But what happens when a test is not statistically significant? Teams often waste hours manually re-analyzing data in third-party tools.&lt;/p&gt;
&lt;p&gt;To solve this we built another AI tool for Opal called the Significance Explorer. It lets you analyze your results using different statistical methods without leaving the platform. This gives you a clearer picture of your experiment&amp;rsquo;s impact and helps you make smarter decisions.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://academy.optimizely.com/student/page/2820737-masterclass-experimentation-evolution-with-ai"&gt;Watch the Masterclass session recording&lt;/a&gt; to get the full insights now.&lt;/p&gt;
&lt;h2&gt;The four big benefits&lt;/h2&gt;
&lt;p&gt;By embracing AI you will see four huge benefits:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Faster time to value&lt;/strong&gt;. Get from idea to results in a fraction of the time.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Scalable growth&lt;/strong&gt;. Run more experiments and drive more growth without hiring more people.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Higher test velocity&lt;/strong&gt;. Remove bottlenecks and dramatically increase your testing rate.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Greater efficiency&lt;/strong&gt;. Automate tedious tasks and free up your team to focus on what matters.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Your turn to evolve&lt;/h2&gt;
&lt;p&gt;The AI revolution is here. The brands that embrace it will dominate. The choice is yours. You can stick with the old ways or you can evolve with AI and make it your superpower.&lt;/p&gt;
&lt;p&gt;Getting started can feel overwhelming. But you do not have to be an expert. Start small. Pick one area of your process and see how AI can help. Then expand from there.&lt;/p&gt;
&lt;p&gt;As I said in this masterclass you just have to get to work. Do the math. Solve one problem then the next. The journey of a thousand miles begins with a single step.&lt;/p&gt;
&lt;p&gt;&lt;img src="/link/fba989c7645240b5aad13e378b30df02.aspx" alt="" width="600" height="339" /&gt;&lt;/p&gt;
&lt;p&gt;Take that step today or contact &lt;a href="https://niteco.com/"&gt;Niteco&lt;/a&gt; for help to get you start off on the right foot.&amp;nbsp;&lt;/p&gt;</summary></entry></feed>