Skip to content

Can AI read your website?

Check any page against ChatGPT, Perplexity, Claude and Google AI.

Free. One page. No account. Results are kept for 24 hours.

See what crawlers see ↓

Example page

Switch views to see what a crawler receives.

Burr espresso grinder

$249.00

4.8 out of 5 stars from 312 reviews

Flat steel burrs and 40 grind settings, from fine espresso to coarse cold brew. Grinds a double shot in about ten seconds.

Four common causes

Why pages go unread

The cause is often technical, and technical causes can be found and fixed.

  • robots.txt blocks AI crawlers

    One old rule, or one added by a plugin, can block GPTBot, OAI-SearchBot, ClaudeBot or PerplexityBot from the whole site. It is easy to miss because people can still visit every page.

    robots.txt
    User-agent: GPTBot
    Disallow: /
    
    User-agent: PerplexityBot
    Disallow: /
  • A CDN or firewall answers with a challenge page

    Bot protection on Cloudflare, Akamai or a WAF often answers automated visitors with a challenge instead of your content. People get through, while crawlers receive a page with nothing to read.

    Response to a crawler
    HTTP/2 403
    cf-mitigated: challenge
    content-type: text/html
  • Content appears only after JavaScript runs

    Many AI crawlers read the raw HTML and do not run scripts. If your text is rendered in the browser, what they receive is an empty shell.

    Raw HTML received
    <body>
      <div id="root"></div>
      <script src="/app.js"></script>
    </body>
  • Pages lack clear structure

    Missing titles and headings, no main landmark and conflicting canonicals make a page harder to interpret. Absent or broken structured data hides who the page belongs to and what it offers.

    Structure scan
    <h1>: missing
    <main>: missing
    JSON-LD: parse error at line 4

Four steps

How it works

From a free page check to a site you know stays readable.

  1. Step 1: Run the free check

    Paste one page address. You get a result right away: whether AI crawlers can reach and read it, most serious findings first.

  2. Step 2: Order a full-site audit

    Up to 500 pages are crawled twice, once as a browser sees them and once as a non-rendering crawler does, then compared page by page.

  3. Step 3: Get a human-reviewed report

    A person reads every report before delivery. It lists findings with evidence, affected pages and fixes in priority order.

  4. Step 4: Fix, then recheck

    Your team applies the fixes, or we can. Request the included recheck from the report page within 14 days to see which findings are resolved, new or still open. A monthly retainer keeps checking after that.

Four scored categories

What we check

The same captured evidence and rule version produce the same score. Each finding includes the observed problem and a way to verify the fix.

How the score is weighted
  • Access 40%
  • Readable content 30%
  • Structure 20%
  • Discovery 10%

Access

Weight 40%

  • Homepage responds, without redirecting to another site or to HTTP
  • robots.txt rules for search crawlers, including OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot, Bingbot and Applebot, and for user-triggered AI fetchers
  • AI training opt-outs, reported but never scored
  • Indexing and snippet controls in headers and HTML
  • CDN or WAF challenge page detected
  • Optional llms.txt file, reported with zero weight

Readable content

Weight 30%

  • Main text present in the raw HTML
  • Raw versus rendered text ratio per page
  • Empty or boilerplate-only pages
  • Language declared

Structure

Weight 20%

  • One meaningful title and a non-empty H1
  • Heading levels in order and a main landmark
  • A declared canonical URL: one value, matching any Link header, not chained and not pointing at an error. A page without one passes
  • Structured data (JSON-LD) parses
  • Labels on links and destinations in language alternates
  • Required JSON-LD fields for Organization, LocalBusiness, Product, FAQPage and BreadcrumbList

Discovery

Weight 10%

  • XML sitemap found, with response checks on sampled entries
  • Broken internal links
  • Duplicate pages without a shared canonical

Measured and reported, never scored

The report also states these measurements for the pages it checked, each with the number of pages examined and the limit of the measurement. A measurement that cannot be made for a page leaves that page out. They describe the site; they carry no points.

  • A fix plan: every failed scored check, ordered by score points per unit of effort
  • Server HTML compared with the browser view: the share of main-content words already present before JavaScript runs, with consent banners and chat widgets left out
  • HTML size, the share of it that is visible text, and an estimated token count for pages written mainly in Latin script
  • In the full audit: time to the response headers, redirect chains and responses too large to read
  • robots.txt, Content-Signal lines, the sitemap, llms.txt and canonical links compared with each other
  • Whether the modification dates a page declares in its sitemap entry, JSON-LD and meta tags agree with each other
  • JSON-LD values compared with the visible text, title, description and image alt text of the same page

Claude, Anthropic’s model, is used for one job: turning verified findings into plain-language explanations and fix steps. It cannot add a finding, remove one or change its severity, and it never sees your raw HTML. A training opt-out never lowers your search-readiness score.

Example report

See what the audit finds

Explore a five-page bakery website. Follow a blocked crawler to the rule behind it, see which URLs return errors and find the first fixes to give your team.

Open the sample report

harborbakery.example

5 URLs checked · 25 checks

Readiness score
75 / 100
Check coverage
100%
  • HighAccess

    robots.txt blocks AI search crawlers

  • MediumReadable content

    Page is empty or holds only navigation and footer

  • MediumDiscovery

    Internal links point to pages that return errors

Illustrative website. The score and findings come from its saved pages.

Five offers

Offers and pricing

Audits are paid before the crawl runs and check up to 500 pages. On a larger site the report shows how many of the pages found were checked, so ask us first. Fixes are always quoted from the audit, never bundled blind.

  • Free page check

    $0one page

    One public URL, instant result, no account.

  • Full-site readiness audit

    Recommended first step

    from $300per site

    Up to 500 pages, deterministic checks, human review, a report page and PDF with evidence and prioritized fixes, and one recheck you request from the report within 14 days. Typically $300 to $500.

  • Fix implementation

    from $800per site

    Robots, CDN and WAF rules, content moved into server HTML, schema, llms.txt and template fixes. Quoted from your audit, typically $800 to $2,000.

  • Monthly retainer

    from $150per site, per month

    Monthly recheck, a diff against the last report, and a verified AI-crawler visit report from your server or Cloudflare logs. Typically $150 to $300.

  • Agency wholesale audit

    from $150per audit

    For SEO consultants and agencies. The same audit, delivered to you to resell. Typically $150 to $250.

    • White-label: your name, logo and colours on the report page and PDF, with no Crawlito branding
    • Request a fix on the report emails you, or is hidden; your clients never contact us
    • Agencies typically resell it at $500 to $1,000 per site
    • Read by a person before delivery, so you can send it as is

Prices in US dollars. Payment by invoice through Wise, Payoneer or a Stripe payment link.

Eight questions

Questions

Will fixing these issues get my site cited by ChatGPT or Perplexity?

No one can promise that, and we do not. An audit removes technical barriers that stop AI tools from reaching and reading your pages. Whether an assistant then cites you depends on many things outside your site.

What does the free check cover?

One public page: its status and redirects, the robots.txt rules for each AI crawler, robots meta and headers, whether the main text is in the raw HTML, and basic structure. It also reports measurements that never change the score, such as the size of the HTML, how much of the page's main content is in the server HTML before JavaScript runs, and whether the site's crawler instructions agree with each other. Response times and redirect chains are reported in the full audit only. The full audit runs the same checks across up to 500 pages and compares raw and rendered versions of each.

Is the score generated by AI?

No. A fixed set of rules produces every finding and the score, so the same site gives the same result. Claude only writes the plain-language explanations and fix steps from findings the rules have already verified.

Do I need to give you access to my site or server?

No. We read what any public visitor or crawler can read. For the monthly retainer's crawler visit report, you share server or Cloudflare logs if you want that part.

How long does an audit take?

The crawl starts once the invoice is paid and is capped at 500 pages; on a larger site the report says how many of the pages found were checked. A person then reviews the report before delivery. We confirm the delivery date when we send the invoice.

What if my site blocks your crawler?

Then the report says so. When we cannot fetch enough of the site, we show Incomplete or Access blocked instead of a score, together with what blocked us. If your robots.txt disallows CrawlitoBot by name, Crawlito respects it and stops before fetching any page; the About CrawlitoBot page explains how.

We block GPTBot on purpose. Will that hurt our score?

No. We label every crawler by purpose: search, training or user-triggered. Opting out of training never lowers your search-readiness score.

Can I resell the audit to my clients?

Yes. The agency wholesale audit carries your name, logo and colours instead of Crawlito's, on the report page and the PDF, and is delivered to you to send on. Request a fix on the report emails you, or you can have it hidden, so your clients never contact us.

Start with one page. The check is free and needs no account.

Check a page