Google AI Overviews requirements: what your site actually needs
Google says AI Overviews and AI Mode need nothing beyond normal Search eligibility. Here is what that means, which controls apply and how to check each.
By Crawlito · 9 min read
In short
- Google says a page must be indexed and eligible to show in Search with a snippet to be a supporting link in AI Overviews or AI Mode, with no additional technical requirements.
- AI Overviews and AI Mode are part of Google Search, so Googlebot rules in robots.txt control crawling for them, not Google-Extended.
- nosnippet, data-nosnippet and max-snippet limit what Google may use as direct input for AI Overviews and AI Mode, and noindex removes the page from Search entirely.
- Google says no llms.txt, AI text files, special markup or special schema.org structured data are needed to appear in its AI features.
- AI Overviews and AI Mode traffic is counted in the Search Console Performance report under the Web search type, with no separate report.
On this page
- What Google says AI Overviews and AI Mode require
- Requirement 1: Googlebot must be able to crawl the page
- Requirement 2: the page must be indexable
- Requirement 3: the page must be eligible for a snippet
- Why Google-Extended does not control AI Overviews
- What you don't need for AI Overviews
- How to measure AI Overviews traffic
- Frequently asked questions
- Check the three conditions on your own pages
Google's answer is short: to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet. Google's AI features documentation says there are no additional technical requirements beyond that. So the AI Overviews requirements are the ordinary Search requirements: Googlebot can fetch the page, the page can be indexed, and your snippet controls allow Google to show and use its text.
That sounds simple, and it is. The work is in making sure nothing on your site quietly breaks one of those three conditions. This guide lists each condition, the controls that affect it, and how to check it.
What Google says AI Overviews and AI Mode require
Google's page on AI features makes four statements worth keeping in front of you:
- A page must be indexed and eligible to appear in Search with a snippet to be a supporting link.
- There are no additional technical requirements and no special optimizations needed.
- You don't need to create new machine-readable files, AI text files or markup, and no special schema.org structured data is required.
- Indexing and serving are not guaranteed, even when everything is in order.
Point 4 matters. Meeting the requirements makes a page eligible. It does not decide whether Google picks it for a given answer. Google also says AI Overviews and AI Mode may use a "query fan-out" technique, issuing several related searches across subtopics, so the pages chosen can come from searches beyond the one the user typed.
The baseline that "eligible" refers to is Google's Search technical requirements, which has three items:
| Requirement | What it means in practice | Common way it fails |
|---|---|---|
| Googlebot isn't blocked | robots.txt, the CDN and the firewall let Googlebot fetch the page | A Disallow rule, a WAF challenge, a geo-block |
| The page works | The URL returns HTTP 200 | 4xx or 5xx errors, soft 404s |
| The page has indexable content | Text in a supported file type that doesn't break spam policies | Text only in images, an empty JavaScript shell |
Requirement 1: Googlebot must be able to crawl the page
AI features are built into Search, so Google points to its Googlebot robots.txt rules as the control for crawling for Search, AI features included. There is no separate AI Overviews crawler to allow or block.
A robots.txt that keeps Googlebot out of a section removes that section from AI Overviews along with everything else in Search:
User-agent: Googlebot
Disallow: /guides/Note that Google's robots.txt introduction says a disallowed URL can still be indexed if other pages link to it, but the search result won't have a description. Google cannot read the blocked page, so it has no page text to use. If you meant to keep a page out of Search, use noindex on a crawlable page instead; robots.txt vs meta robots explains the difference.
Robots.txt is only one layer. Google's best practices for AI features also say to allow crawling through any CDN or hosting infrastructure. A bot rule or WAF challenge page that serves Googlebot an interstitial blocks it as effectively as a Disallow line, and it won't show up in your robots.txt.
How to check Googlebot access
- In Search Console, run the URL through URL Inspection and use Test live URL. It reports whether crawling is allowed and shows the HTML Google fetched.
- Open the robots.txt report in Search Console to see the robots.txt files Google found for the top 20 hosts on your site, when each was last crawled, and any warnings or errors.
- In your server or CDN logs, confirm Googlebot requests get a 200. Verify that a request is really from Google with a reverse and forward DNS lookup, as described in Google's verification guide:
host 66.249.66.1
host crawl-66-249-66-1.googlebot.comThe first command should return a pointer to a name such as crawl-66-249-66-1.googlebot.com, and the second should return 66.249.66.1 again. The hostname must end in googlebot.com, google.com or googleusercontent.com, and the forward lookup must return the original IP.
Requirement 2: the page must be indexable
A page with noindex is out of Google Search, which means it is out of AI Overviews and AI Mode too. Google lists noindex as the control for blocking indexing, in the same breath as the AI feature controls.
noindex can arrive in two places, and you need to check both:
<meta name="robots" content="noindex">
<meta name="googlebot" content="noindex">HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
X-Robots-Tag: noindexThe header version is easy to miss because it never appears in the page source. It is often set by a CDN rule, a framework config or a plugin, and one source is a setting copied over from staging; the leftover noindex covers how that happens.
Indexable also means the text is there for Google to read. Google renders JavaScript with an evergreen Chromium, but rendering is queued and can take longer than crawling, and Google's own JavaScript SEO guide still calls server-side rendering or pre-rendering a great idea. Google's AI features best practices add: keep important content available in text form.
How to check indexability
Fetch the headers and the raw HTML and look for robots directives. The first command prints the status line and any X-Robots-Tag header; the second prints any robots or googlebot meta tag in the raw HTML:
URL="https://example.com/guides/pricing"
curl -s -o /dev/null -D - "$URL" | grep -iE "^(HTTP|x-robots-tag)"
curl -s "$URL" | grep -ioE '<meta[^>]+name="(robots|googlebot)"[^>]*>'A curl from your own machine is not Googlebot, and a firewall may treat it differently, so treat this as a first pass. URL Inspection in Search Console is the authoritative check: it tells you whether the page is indexed and, if not, why.
Requirement 3: the page must be eligible for a snippet
This is the condition most closely tied to how AI features use your text. Google's robots meta tag documentation says the nosnippet rule also prevents content from being used as a direct input for AI Overviews and AI Mode, and that max-snippet also limits how much content may be used as a direct input for them.
Here is how the controls compare:
| Control | Where it goes | Effect in Search | Effect on AI Overviews and AI Mode |
|---|---|---|---|
noindex | meta robots or X-Robots-Tag | Page is not indexed | Page cannot be a supporting link |
nosnippet | meta robots or X-Robots-Tag | No text snippet shown | Content not used as direct input |
max-snippet:[number] | meta robots or X-Robots-Tag | Snippet capped at that many characters | Caps how much content is used as direct input |
data-nosnippet | HTML attribute on span, div or section | That part is left out of snippets | Google lists it as a control for what AI features show |
Googlebot Disallow | robots.txt | Page content not crawled | Page content not available |
Google-Extended Disallow | robots.txt | No effect on Search inclusion or ranking | Not the control for these features |
Some examples:
<!-- Keep the page in Search but allow no snippet or AI input -->
<meta name="robots" content="nosnippet">
<!-- Allow a short snippet only -->
<meta name="robots" content="max-snippet:50">
<!-- Let Google choose the length, the same as leaving the rule out -->
<meta name="robots" content="max-snippet:-1">
<!-- Exclude one block of the page -->
<div data-nosnippet>Member pricing is shown after sign-in.</div>data-nosnippet is supported only on span, div and section elements. Google's documentation marks it as not valid on other elements, so wrap a p or li in a span or div rather than putting the attribute on it directly.
The practical trap is that these rules are often set for one reason and have a second effect. A max-snippet:0 added years ago to stop text appearing in results now also limits AI input, because Google's documentation says max-snippet:0 is equivalent to nosnippet. If your pages carry a site-wide snippet limit, decide whether you still want it. Our guide to noindex, nosnippet and X-Robots-Tag goes through each directive in more detail.
How to check snippet controls
Use the same curl commands as above and look for nosnippet or max-snippet in the meta tags and the X-Robots-Tag header. Then search the raw HTML for data-nosnippet to see which blocks are excluded:
curl -s "$URL" | grep -o 'data-nosnippet' | wc -lIf the count is high, check that the attribute is not wrapped around your main content by a template.
Why Google-Extended does not control AI Overviews
This is a common mistake in AI Overviews advice. Google-Extended is a robots.txt token that controls whether content Google crawls may be used to train future generations of the Gemini models behind Gemini Apps and the Vertex AI API for Gemini, and for grounding, such as Grounding with Google Search on Vertex AI. Google's crawler overview states that Google-Extended does not affect whether a site is included in Google Search and is not used as a ranking signal. The AI features page points to Google-Extended only for limiting AI training and grounding in some of Google's other systems.
So this rule:
User-agent: Google-Extended
Disallow: /does not remove a site from AI Overviews or AI Mode. Google-Extended also has no user-agent string of its own. You will never see it in your logs, because crawling is done by existing Google user agents and the token is read only from robots.txt. Google-Extended explained covers what it does control.
If you want to limit how your text is used in AI Overviews, the documented options are the snippet controls in the previous section, or noindex if you want the page out of Search altogether.
What you don't need for AI Overviews
Google is direct about what is not required. Its AI features page says you don't need new machine-readable files, AI text files or markup, and no special schema.org structured data. Its AI optimization guide adds that Google Search ignores llms.txt files, so maintaining one for other services neither helps nor harms visibility in Google Search.
That does not make structured data useless. Google's best practices say structured data should match the visible text on the page, and markup can still support other Search features. It is just not an entry ticket for AI Overviews. Eight AI search myths covers this and similar claims.
The rest of Google's best practices list for AI features is ordinary site hygiene:
- Allow crawling in robots.txt and through your CDN or hosting.
- Make content findable through internal links.
- Provide a good page experience.
- Keep important content in text form, supported by images and video where useful.
- Keep Merchant Center and Business Profile information up to date where they apply.
- Verify the site in Search Console to find and diagnose technical issues.
How to measure AI Overviews traffic
There is no separate AI Overviews or AI Mode report in Search Console at the time of writing. Google says traffic from sites shown in AI features is counted in the overall search traffic in the Performance report, under the Web search type, and that AI Mode data has counted toward Performance report totals since June 16, 2025. You can see that clicks came from Google Search, but not which ones came from an AI Overview.
When you change a control, give it time. Google says recrawling can take anywhere from several days to several months, and you can request a recrawl of a specific URL through URL Inspection.
Frequently asked questions
Does blocking Google-Extended remove my site from AI Overviews?
No. Google states that Google-Extended does not affect inclusion in Google Search, and AI Overviews and AI Mode are part of Search. The controls for them are Googlebot rules in robots.txt, noindex and the snippet controls.
Do I need llms.txt or special schema for AI Overviews?
No. Google says no new machine-readable files, AI text files, markup or special schema.org structured data are needed, and that Google Search ignores llms.txt.
Can I opt out of AI Overviews and stay in regular search results?
Google documents no switch that applies only to AI features. nosnippet and max-snippet limit both regular snippets and what is used as direct input for AI Overviews and AI Mode, and data-nosnippet does the same for parts of a page.
Why is my indexed page never shown as a supporting link?
Being eligible does not mean being selected. Google says indexing and serving are not guaranteed, so once the page is crawlable, indexed and snippet-eligible, the remaining factors are outside these technical checks.
Where can I see AI Overviews traffic in Search Console?
In the Performance report under the Web search type, counted together with other search traffic. There is no separate AI Overviews report.
Check the three conditions on your own pages
Every AI Overviews requirement Google documents comes down to three things: Googlebot can fetch the page, the page is indexable, and nothing stops a snippet. A sensible next step is to run the free check on one important page. It covers a single page and needs no account, and it gives you a starting point before you work through URL Inspection and the commands above.
Filed under Strategy and audits. Topics: ai-overviews, google, google-extended, meta-robots, robots-txt, checklist