Skip to content

Cloudflare Agent Readiness score: what isitagentready checks

What the Cloudflare Agent Readiness score on isitagentready.com checks, how the level and percentage work, and which failed checks matter for your kind of site.

By Crawlito · 9 min read

In short

  • Cloudflare's Agent Readiness scanner at isitagentready.com groups its checks into Discoverability, Content, Bot Access Control, API/Auth/MCP discovery and Commerce, and the commerce checks do not count toward the score.
  • The result shows a level from 0 (Not Ready) to 5 (Agent-Native) plus a percentage, and level 4 requires an MCP Server Card, A2A Agent Card, Agent Skills index or API Catalog that a blog or brochure site has no reason to publish.
  • Choosing the Content Site profile limits the scan to seven checks: robots.txt, sitemap, Link headers, DNS-AID, Markdown negotiation, AI bot rules and Content Signals.
  • Of the scored checks, only the robots.txt rules concern whether AI search crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot are allowed to fetch your pages.
  • Cloudflare's April 2026 launch post reported MCP Server Cards and API Catalogs on fewer than 15 sites among a filtered set of the 200,000 most visited domains, so failing those checks is the norm.
On this page

The Cloudflare Agent Readiness score, shown on isitagentready.com, measures how many emerging agent standards a site publishes: robots.txt and sitemaps, Markdown content negotiation, Content Signals, Web Bot Auth, API catalogs, OAuth metadata, MCP server cards, WebMCP and more. A low score usually means you have not published files that only API products, apps and agent platforms need. For a content site, the checks that matter today are a small subset, and of all the checks only the robots.txt rules concern whether AI search crawlers are allowed to fetch your pages.

This post lists what the scanner checks, how its level and percentage are calculated, and which failures to fix first for your type of site. Everything here comes from Cloudflare's launch post of 17 April 2026 and the scanner's own published skill files and page code, at the time of writing. The tool changes often, so recheck before you plan work around a single check.

What the Cloudflare Agent Readiness score is

Cloudflare launched the scanner on 17 April 2026. You enter a URL, Cloudflare makes requests to the site and reports which standards it supports. For each failed check it shows a prompt you can paste into a coding agent. The page carries a disclaimer that these improvement instructions are AI-generated and may contain mistakes, so treat them as a draft.

The same checks are available in other places:

  • Cloudflare Radar URL Scanner has an Agent Readiness tab, and the URL Scanner API accepts "options": {"agentReadiness": true}, according to the launch post.
  • AI Crawl Control in the Cloudflare dashboard renamed its Robots.txt tab to Directives and links to the check, per the changelog entry.
  • An API and MCP server. The scanner's own skill file documents POST https://isitagentready.com/api/scan and an MCP server at https://isitagentready.com/mcp with a tool called scan_site.
bash
curl -s -X POST https://isitagentready.com/api/scan \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.example.com", "format": "agent"}'

The agent format returns Markdown with failing checks and fixes; the default json format returns each check's status as pass, fail or unableToCheck.

Level 0 to 5 versus the percentage

The result card shows two numbers, and they answer different questions.

The level is a ladder. The scanner's skill file defines it like this:

LevelNameWhat it takes
0Not ReadyFewer than 2 of robots.txt, sitemap, Link headers
1Basic Web Presence2 of those 3
2Bot-AwareLevel 1 plus AI bot rules in robots.txt and Content Signals
3Agent-ReadableLevel 2 plus Markdown content negotiation
4Agent-IntegratedLevel 3 plus one of MCP Server Card, A2A Agent Card, Agent Skills index or API Catalog
5Agent-NativeLevel 4 plus 2 of 3: Web Bot Auth, all integrations, auth metadata (OAuth or Auth.md)

The percentage is broader. In the scanner's page code at the time of writing, each enabled check counts equally, the overall figure is passed checks divided by counted checks across the four scored categories, and the Commerce category is left out. The Personal Agent Protocol check is marked informational and does not affect either number.

The practical consequence: a well-built blog can reach level 3 at most without publishing API or agent metadata. Levels 4 and 5 describe sites that offer machine interfaces, not sites that are easier to read.

What each check tests

The scanner's check list has grown since launch. At the time of writing it shows these checks in the default scan, with the paths and specs its fix prompts name:

CategoryCheckWhat it looks for
Discoverabilityrobots.txtA plain-text /robots.txt returning 200 (RFC 9309)
DiscoverabilitySitemapA sitemap.xml, referenced from robots.txt
DiscoverabilityLink headersLink response headers on the homepage (RFC 8288), such as rel="api-catalog"
DiscoverabilityDNS-AIDSVCB records such as _index._agents.example.com (an IETF draft)
ContentMarkdown negotiationtext/markdown returned for Accept: text/markdown
Bot Access ControlAI bot rulesUser-agent groups for crawlers like OAI-SearchBot, Claude-SearchBot, GPTBot and ClaudeBot
Bot Access ControlContent SignalsA Content-Signal line in robots.txt
Bot Access ControlWeb Bot AuthA key directory at /.well-known/http-message-signatures-directory
API, Auth, MCP and Skill discoveryAPI Catalog/.well-known/api-catalog (RFC 9727)
API, Auth, MCP and Skill discoveryOAuth discovery/.well-known/openid-configuration or /.well-known/oauth-authorization-server
API, Auth, MCP and Skill discoveryOAuth Protected Resource/.well-known/oauth-protected-resource (RFC 9728)
API, Auth, MCP and Skill discoveryAuth.md/auth.md agent registration metadata
API, Auth, MCP and Skill discoveryMCP Server Card/.well-known/mcp/server-card.json, a draft MCP proposal
API, Auth, MCP and Skill discoveryAgent Skills/.well-known/agent-skills/index.json
API, Auth, MCP and Skill discoveryWebMCPTools registered in the browser through the WebMCP API
API, Auth, MCP and Skill discoveryARD/.well-known/ai-catalog.json
Commerce (unscored)x402, MPP, UCP, ACPAgent payment and commerce discovery

The A2A Agent Card check (/.well-known/agent-card.json) exists but is not enabled in the default scan. The launch post said llms.txt was not checked by default, and at the time of writing it does not appear in the scanner's list of selectable checks at all, so it cannot affect the score.

Which checks affect AI search crawlers today

Most of the scored checks are about agents that call your APIs or act in a browser, not the crawlers that ChatGPT search, Claude and Perplexity use to read pages. Split them before you fix anything.

Checks that affect AI search access now

  • robots.txt and AI bot rules. Of the scanner's checks, these are the only ones about whether AI search crawlers are allowed in. The scanner's own fix prompt makes the key point: search crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot fetch the pages assistants use, while blocking training crawlers such as GPTBot, ClaudeBot or Google-Extended does not affect AI search. A pass does not mean those crawlers are allowed, though. The check's skill file says a User-agent: * group with rules also passes, so read what your groups actually say.
  • Sitemap. A sitemap helps any crawler find your URLs, and referencing it from robots.txt is a long-standing convention.

Checks that state preferences or offer extra formats

  • Content Signals publish how you want content used. At the time of writing, the major AI operators' crawler documentation does not say they act on them; see Content Signals in robots.txt.
  • Markdown negotiation serves Markdown to clients that ask for it. The launch post cites a February 2026 test by Checkly in which only Claude Code, OpenCode and Cursor, of seven agents tested, sent Accept: text/markdown by default. Those are coding agents, not search crawlers. Serving Markdown to AI agents covers the setup and its caching and cloaking pitfalls.

Checks for API, app and agent sites

API Catalog, OAuth discovery, OAuth Protected Resource, Auth.md, MCP Server Card, Agent Skills, ARD, A2A and DNS-AID describe machine interfaces. If you have no public API, no login for agents and no MCP server, there is nothing true to publish. WebMCP is the same idea in the browser; see WebMCP explained. Web Bot Auth is about signing requests your own bots send: Cloudflare's post says a site that only serves content and does not make requests to other sites does not need it.

Pick the right site type before you read the score

The scanner's Customize scan panel has three site types:

Site typeChecks included
All ChecksEvery default check, including commerce
Content Siterobots.txt, Sitemap, Link headers, DNS-AID, Markdown negotiation, AI bot rules, Content Signals
API / ApplicationAll checks except commerce

For a blog, documentation site or marketing site, run it as Content Site. The percentage then reflects seven checks instead of the full list, and you stop being measured against OAuth and MCP files you should not publish.

Even inside the Content Site set, two checks rarely matter for AI search. Link headers on the homepage are useful when you point at an API catalog or documentation, and DNS-AID is an IETF draft for publishing agent endpoints in DNS. Neither is documented by OpenAI, Anthropic, Perplexity or Google as something their search crawlers read.

How to raise the score without wasted work

Work in this order, and verify each step yourself rather than trusting the scanner alone:

  1. Confirm AI search crawlers can fetch your pages. Check that robots.txt allows OAI-SearchBot, Claude-SearchBot and PerplexityBot, and that no firewall or bot setting challenges them.
  2. Publish a sitemap and reference it. Add a Sitemap: line to robots.txt.
  3. Write explicit AI crawler groups that match your policy. For example:
User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
Disallow: /

Sitemap: https://www.example.com/sitemap.xml
  1. Add a Content-Signal line if you want to state a preference. Make it agree with your crawler rules.
  2. Consider Markdown negotiation if agents that ask for it matter to you. On Cloudflare, Markdown for Agents does it at the edge. Unless your origin sends its own Content-Signal header, it adds Content-Signal: ai-train=yes, search=yes, ai-input=yes to the Markdown response, so check that this does not contradict your robots.txt.
  3. Stop there unless you run an API or app. If you do, an API catalog is the simplest next step. RFC 9727 says a GET to /.well-known/api-catalog returns the catalog in the application/linkset+json format and a HEAD returns a Link header with the api-catalog relation. A HEAD response could look like this:
http
HTTP/1.1 200 OK
Content-Type: application/linkset+json; profile="https://www.rfc-editor.org/info/rfc9727"
Link: </.well-known/api-catalog>; rel="api-catalog"

For context, the launch post reported a Cloudflare Radar scan of the 200,000 most visited domains, with categories such as redirects, ad servers and tunneling services filtered out. At launch it found robots.txt on 78% of them, Content Signals on 4%, passing Markdown negotiation on 3.9%, and MCP Server Cards and API Catalogs on fewer than 15 sites in total. A low level is the norm right now, not a sign that something is broken.

How it differs from an AI search readiness check

The Agent Readiness score asks whether you publish agent standards. It does not test whether a page's text is in the raw HTML, whether a firewall serves AI crawlers a challenge page, whether pages carry noindex, or whether your sitemap URLs actually resolve. Those decide whether AI search tools can read you today. Crawlito's score is built around them instead; how the Crawlito score works explains its categories and weights. For machine interfaces beyond the checklist, NLWeb and MCP for websites covers when they are worth building.

Neither score predicts whether an assistant will cite you. Both measure technical conditions you control.

Frequently asked questions

Why is my Agent Readiness level so low?

Level 4 requires an MCP Server Card, A2A Agent Card, Agent Skills index or API Catalog, and level 5 adds Web Bot Auth or auth metadata, which most content sites do not have. Run the scan with the Content Site profile and look at robots.txt, sitemap and AI bot rules first.

Does a higher Agent Readiness score help me appear in ChatGPT or Perplexity?

Not directly. Only the robots.txt access rules affect whether their search crawlers can fetch your pages, and no AI search operator documents using the other checks as a ranking or citation factor.

Do commerce checks lower the score?

No. Cloudflare says x402, Universal Commerce Protocol and Agentic Commerce Protocol do not count toward the score, and the scanner's code excludes the Commerce category from the percentage.

Does the scanner check llms.txt?

Cloudflare's launch post says llms.txt is not checked by default, and at the time of writing it is not among the scanner's selectable checks, so it does not affect the score. Our llms.txt guide covers when it is still worth publishing.

Should I publish an MCP server card just to pass the check?

Only if you run an MCP server. A card that points to nothing gives agents false information and does nothing for AI search.

Check what AI crawlers actually get

Once your robots.txt rules are right, test the part the Agent Readiness scan does not cover: whether the robots.txt rules, the response and the raw HTML of a real page let AI search crawlers in and give them the content. The free check does that for one page with no account.

Filed under Strategy and audits. Topics: cloudflare, ai-agents, audits, llms-txt, web-bot-auth