Is your website agent-ready? Using Cloudflare's readiness tool to assess your website
Part one: the visitor you didn't design for
For twenty-five years we have built websites for two audiences: people, and the search engines that send people to us. Every navigation decision, every page template and every SEO checklist assumes that, sooner or later, a human being is going to look at the page.
That assumption is breaking. A growing share of the research, comparison and shortlisting that used to happen on our sites now happens inside an AI assistant. A customer asks ChatGPT, Claude, Gemini or Copilot to compare three products, summarise a returns policy or find a supplier that meets a specification. The assistant goes out, reads the web on their behalf, and comes back with an answer. Sometimes that answer includes a link to you. Often the customer acts on it without ever seeing your homepage.
In my last post on WebMCP I looked at the far end of this journey: giving agents structured actions they can take on your site. This post is about the step that comes before any of that. Before an agent can act on your site, it has to be able to find it, read it and understand what it is allowed to do with it. That is what being agent-ready means, and most sites, including plenty of well-built Optimizely sites, are not there yet.
Why this is a leadership conversation, not just a developer ticket
It is tempting to file this under "technical SEO" and hand it to the development team. That undersells it. Agent-readiness raises questions that belong with marketing, digital and legal leadership as much as with engineering:
- Do we want AI assistants to recommend us? If so, the crawlers that feed AI answers need to be able to reach and read our content.
- Are we happy for our content to train AI models? That is a separate decision from appearing in AI answers, and most organisations have never made it explicitly.
- What does an agent see when it reads our product and service pages? If it can only parse half the page, it will fill the gaps from somewhere else, possibly a competitor or an out-of-date review.
- Who owns this? Today, in most organisations, the honest answer is "nobody", which means the decision is being made by default settings.
The sites that will do well in the next few years are the ones that make these choices deliberately rather than discovering them by accident.
The business benefits of getting it right
01
Be found
Appear in the AI answers where customers now research and shortlist.
02
Be described accurately
Content agents can read means they get your story right.
03
Own your AI policy
Decide search, AI answers and training separately, in machine-readable form.
04
Serve agents for less
Lightweight Markdown is cheaper to serve and easier to use accurately.
05
Ready for agentic commerce
Discovery and readability come before any agent can check out.
06
Measure progress
A 0–5 readiness score leadership can baseline, target and track.
07
Quick wins first
The early levels are mostly configuration, not new development.
→
Start today
Scan your site in under a minute and share the result internally.
Treat it like mobile-friendly in 2012
There is a useful parallel with the shift to mobile. For a few years, a mobile-friendly site was a differentiator. Then it became table stakes, and the sites that hadn't adapted were quietly penalised. Agent-readiness is on the same curve, and we are early on it. The standards are still settling and the scoring will change, but the direction is clear. The cost of starting now is low; the cost of starting late is being invisible to a growing share of your customers' journeys.
Part two: how to measure it and fix it
The good news is that you no longer have to guess where you stand. Two tools make this practical.
Step 1: Scan your site with isitagentready.com
Is It Agent Ready? is a free scanner from Cloudflare. Enter a URL and it runs a set of checks against the emerging standards for AI agents, grouped into five categories:
| Category | What it looks at |
|---|---|
| Discoverability | robots.txt, sitemap, Link response headers, DNS for AI Discovery (DNS-AID) |
| Content accessibility | Whether the site returns Markdown when an agent asks for it |
| Bot access control | AI bot rules in robots.txt, Content Signals, Web Bot Auth |
| API, auth, MCP and skill discovery | API Catalog, OAuth discovery, MCP Server Card, A2A Agent Card, Agent Skills index, WebMCP and related standards |
| Commerce (scored only for e-commerce sites) | Agentic payment and commerce protocols such as x402, MPP, UCP and ACP |
The result is a readiness level from 0 to 5:
| Level | Name | What it takes |
|---|---|---|
| 0 | Not Ready | Fails the basic checks |
| 1 | Basic Web Presence | Two of: robots.txt, sitemap, Link headers |
| 2 | Bot-Aware | Level 1, plus AI bot rules and Content Signals in robots.txt |
| 3 | Agent-Readable | Level 2, plus Markdown content negotiation |
| 4 | Agent-Integrated | Level 3, plus one of: MCP Server Card, A2A Agent Card, Agent Skills index, API Catalog |
| 5 | Agent-Native | Level 4, plus two of: Web Bot Auth, all integrations, auth metadata |
The most useful part of the report is the "how to implement" section: for each failing check it gives you a ready-made prompt to paste into a coding agent such as Claude Code, Cursor or GitHub Copilot. As Cloudflare points out on the page, these are AI-generated recommendations, so treat them as a starting point and review them like any other change.
My advice: run it against your production site today. It takes under a minute, and the result is usually a useful conversation starter with stakeholders.
Step 2: Bring it into your workflow with the agent-ready-cloudflare skill
The web scanner is great for a one-off check. For ongoing work, and especially if you look after more than one site, the agent-ready-cloudflare skill is the more powerful option. It is a community-published Agent Skill that wraps the same scanner so you can drive it from Claude (or any agent that supports skills).
What it gives you:
- Scanning from your agent. Ask "scan example.com for agent readiness" and the skill calls the scanner's API and produces a structured Markdown report: your level, a pass/fail summary per category, and what you need for the next level.
- Fix prompts for every failing check. The scanner's API only returns an implementation prompt for the next level up. The skill includes templates for all the checks, so its report rebuilds the full set of web-UI style prompts (goal, issue, fix, links to the relevant RFCs and specs) for everything that fails.
- 25 implementation sub-skills. Each standard (robots.txt, Content Signals, Markdown negotiation, Link headers, API Catalog, MCP Server Card, Agent Skills index, WebMCP and so on) has its own short guide covering requirements, the Cloudflare shortcut where one exists, and how to validate.
- Batch scanning. It documents how to scan a list of domains with a polite delay between requests, which is ideal for agencies, multi-brand groups or anyone running several Optimizely sites.
The scanner is also exposed as an MCP server (https://isitagentready.com/mcp, with a scan_site tool), so you can connect it directly to an MCP-capable agent if you prefer.
A practical workflow looks like this:
- Install the skill in Claude.
- Ask it to scan each of your sites and save the reports.
- Pick the target level for each site (Level 3 is a sensible first goal).
- Hand the generated fix prompts to your development team, or to a coding agent working in your solution, and review the output.
- Re-scan to confirm the checks now pass, and repeat on a regular cadence.
Start today
Run the scan, share the result, and agree who owns your AI content policy. The report and the skill will tell you exactly what to fix; the harder and more valuable work is deciding what you want agents to be able to do with your site.
How can Niteco help you?
Niteco is the largest global certified Optimizely CMS and Commerce partner, with deep expertise in the Optimizely Agent Platform.
#1
Optimizely
The largest global partner. Powered by technical excellence and innovation. Accelerated by AI.
400+
AI-trained experts
Certified by leading platforms like Optimizely, Microsoft, Contentful & Adobe.
2000+
Successful projects
We have vast experience from years of delivering digital platforms for clients in diverse sectors.
9
Optimizely MVPs
Largest team with OMVPs. Meaning the best & most experienced Optimizely minds will work with you.
As Optimizely's largest certified partner in Optimizely CMS, Commerce, and the agent platform, we can help you with any of your strategic initiatives globally around content, commerce, or your organisational AI.
If you would like help making your Optimizely site agent-ready, get in touch with us at Niteco: https://niteco.com/contact-us/
Comments