Bing AI grounding controls: the meta tag that replaces NoArchive
Bing's new AI grounding meta tag and X-Robots-Tag values replace NoArchive and NoCache. What allow, disallowextended and disallow do, and the 2027 deadlines.
By Crawlito · 8 min read
In short
- Bing now separates AI training from AI grounding: grounding is set per page with a Bingbot meta tag or X-Robots-Tag header, using allow, disallowextended or disallow.
- disallowextended keeps a page available to Microsoft's own AI experiences, such as Copilot and Microsoft 365, but excludes grounding services Microsoft provides to third parties, such as Web IQ and Foundry.
- Microsoft starts honouring the new preferences on 1 February 2027, and after 1 March 2027 NoArchive and NoCache are no longer read as training or grounding preferences.
- A temporary grounding preference in Bing Webmaster Tools opens on 15 December 2026, only for sites using NoArchive or NoCache on that date, and expires at the earlier of a recrawl or 30 September 2027.
- Once Microsoft detects any new AI Training or AI Grounding preference on a site, it disregards every NoArchive and NoCache tag left on that site.
On this page
- What changes from NoArchive and NoCache
- The three AI grounding values
- How to set AI grounding with a meta tag or X-Robots-Tag
- The ai-training value and how it interacts with robots.txt
- The migration timeline and the 15 December window
- What to do now, by starting point
- How to check the tags are being served
- Frequently asked questions
- Check that Bingbot can read your preferences
Bing is replacing noarchive and nocache as its AI controls with two new, independent preferences: AI Training and AI Grounding. Grounding, meaning whether a page may inform generated answers in Bing, Copilot and Microsoft 365, is set only per page, with <meta name="Bingbot" content="ai-grounding:disallow"> or the matching X-Robots-Tag header. Microsoft starts honouring the new values on 1 February 2027 and stops reading noarchive and nocache for AI use after 1 March 2027. If you rely on those old tags today, you need to publish the new ones by that date.
Unless another source is linked, everything below comes from Microsoft's announcement, by Krishna Madhavan, Your content, your choice: clearer preferences for training and grounding, published on the Bing Search blog on 30 October 2026 and read on 15 November 2026. Microsoft says a companion FAQ sits in the Bing Webmaster Tools help documentation. For how Bingbot and Copilot fit together in general, see Bingbot and AI search.
What changes from NoArchive and NoCache
Bing announced noarchive and nocache as AI controls in September 2023, for what was then Bing Chat. The two values did two jobs at once: noarchive kept a page out of chat answers and out of training, and nocache limited both to the URL, title and snippet. You could not refuse training while staying in answers, or the reverse.
The new scheme splits the two jobs:
| Old scheme (2023) | New scheme (2026) | |
|---|---|---|
| Controls | noarchive, nocache | AI Training, AI Grounding |
| Training and answers | Tied together | Independent of each other |
| Where training is set | Page only | robots.txt (site or path) and page |
| Where grounding is set | Page only | Page only |
| Grounding levels | Excluded or snippet only | Allow, DisallowExtended, Disallow |
| Third-party grounding | Not addressed | Can be excluded separately |
Microsoft says the two are independent: your training setting does not change your grounding setting, and vice versa. Both default to Allow when nothing applicable is published.
The three AI grounding values
Grounding is the use of your page to inform a generated response. Microsoft defines three values:
| Value | Meaning in Microsoft's post |
|---|---|
allow (default) | Content may help inform generated responses |
disallowextended | Grounding limited to Microsoft-operated experiences, such as AI Mode, AI Overview, Copilot and Microsoft 365; grounding services Microsoft provides to third parties, such as Web IQ and grounding through API access in Foundry, are excluded |
disallow | Content is not used to inform generated responses |
disallowextended is the value with no equivalent elsewhere. It is for owners who are happy to appear in Copilot answers but do not want their pages feeding AI products that other companies build on Microsoft's grounding services. The old tags could not express that distinction.
Microsoft is direct about the cost: if you select DisallowExtended or Disallow for grounding, "you will see reduced brand visibility in AI experiences." Allow does not promise the opposite. Microsoft also says none of these choices guarantees inclusion, traffic or payment, and that they do not create any licensing arrangement.
How to set AI grounding with a meta tag or X-Robots-Tag
The grounding preference uses the Bingbot meta tag name or an X-Robots-Tag header that names Bingbot. Microsoft's post shows the disallow form and tells you to use ai-grounding:disallowextended for the middle level:
<meta name="Bingbot" content="ai-grounding:disallow">
<meta name="Bingbot" content="ai-grounding:disallowextended">X-Robots-Tag: Bingbot: ai-grounding:disallowSubstitute ai-grounding:disallowextended in the header for the middle level. Microsoft's post does not discuss file types, but a header is the usual way to label PDFs and other files without an HTML head; the mechanics are in X-Robots-Tag for PDFs and other files.
Points the post settles, and one it does not:
- There is no site-wide grounding setting. Microsoft states: "There is no site-wide robots.txt grounding preference." To restrict grounding across a site you have to send the tag or header on every page, which is usually a template or server change rather than an editorial one.
- The tag names Bingbot only. Microsoft's examples use
name="Bingbot", notname="robots", so other search engines are not addressed by it. The post does not say whethername="robots"would also work, so use the documented form. - Combined values are not documented. Each example sets one preference. Microsoft shows no syntax for putting
ai-trainingandai-groundingin onecontentattribute, so publish each as its own tag or header line, as in the examples.
To refuse both training and grounding on one page with the documented forms:
<meta name="Bingbot" content="ai-training:disallow">
<meta name="Bingbot" content="ai-grounding:disallow">The ai-training value and how it interacts with robots.txt
Training can be set in two places. At page level it uses the same pattern:
<meta name="Bingbot" content="ai-training:disallow">X-Robots-Tag: Bingbot: ai-training:disallowAt site or path level it uses the shared robots.txt token __usage_ai-training, which Microsoft's post links to an IETF Internet-Draft. Writing User-Agent: __usage_ai-training followed by Disallow: / refuses training for the whole site, and a path in place of / refuses it for that section. How the token works, and which other operators read it, is covered in the __usage_ai-training robots.txt token.
Microsoft gives three precedence rules:
- Page beats robots.txt for training. "The new AI Training preference set at a page level overrides the robots.txt setting." So a page tagged
ai-training:disallowinside an allowed site is excluded from training. Microsoft lists Allow and Disallow as the training values but only shows the disallow tag, so test before relying on a page-level allow to carve out an exception. - New preferences switch off the old tags for the whole site. If Microsoft detects new AI Training or AI Grounding preferences on a site, it disregards any
noarchiveandnocacheleft on that site. Adding the new tag to one test page can change how every other page is treated. - Usage preferences never unlock crawling. "An Allow setting does not override a robots.txt rule blocking Bingbot from crawling." A page Bingbot cannot fetch cannot be read at all, whatever its tags say.
None of these controls touches ordinary search. Microsoft says the preferences do not change eligibility for traditional Bing results, and that the Bing search syndication API is not affected by the grounding preference. To leave Bing entirely you still need noindex, which removing a page from AI search covers across operators.
The migration timeline and the 15 December window
Microsoft published four dates:
| Date | What happens |
|---|---|
| 15 December 2026 | Temporary grounding transition preferences open in Bing Webmaster Tools, only for sites using noarchive or nocache as of that date |
| 1 February 2027 | Microsoft begins honouring the new AI Training and AI Grounding preferences as it crawls |
| 1 March 2027 | Deadline to publish page-level grounding preferences and record a temporary grounding preference; training preferences must be published in robots.txt or on pages by this date; after it, noarchive and nocache are no longer read for training or grounding |
| 30 September 2027 | Temporary grounding preferences expire, or earlier if Microsoft recrawls the page and finds its new grounding preference |
Why a temporary setting exists: a tag only takes effect when Bingbot recrawls the page, and Microsoft says it "must crawl the page to discover the setting." Recrawling a large site takes time. The temporary preference in Bing Webmaster Tools covers that gap for grounding. Training has no temporary setting, so it must be published in robots.txt or on pages by 1 March 2027.
The eligibility rule is the detail that can catch people out. The transition preference is only for sites using noarchive or nocache as of 15 December 2026. Based on that wording, a site that removes its old tags before that date, perhaps while tidying up, would not qualify. Keep the old tags in place until you have recorded the temporary preference and published the new tags.
Missing the deadline has a default. Pages using noarchive or nocache with no temporary grounding preference recorded by 1 March 2027 default to Allow for grounding until Microsoft recrawls them. Tags published after that date can still take effect on recrawl, but Microsoft says an earlier recrawl is not guaranteed. IndexNow can prompt a recrawl, but "it does not guarantee a recrawl date."
At the time of writing, the post does not describe where the temporary preference sits in Bing Webmaster Tools or what its options are called, and it does not map noarchive or nocache to a specific new value. Check Bing Webmaster Tools after 15 December rather than relying on screenshots from elsewhere.
What to do now, by starting point
| Current setup | Grounding action | Training action |
|---|---|---|
No noarchive or nocache, happy with defaults | None; both default to Allow | None |
noarchive to stay out of Copilot answers | Record the temporary preference after 15 December, add ai-grounding:disallow to the pages by 1 March 2027 | Add __usage_ai-training rules or ai-training:disallow by 1 March 2027 |
nocache to limit answers to snippets | Choose a new level; there is no snippet-only grounding value, and the post does not map nocache to one | Same as above, if you still want to refuse training |
| Want Copilot answers but not training | Leave grounding at Allow | Disallow training in robots.txt or per page |
| Want Microsoft's AI but not third-party grounding | ai-grounding:disallowextended on each page | Your choice |
The fourth row is the Bing version of allowing AI search while blocking AI training, now possible without an all-or-nothing tag.
How to check the tags are being served
A tag only counts if Bingbot can fetch the page and see it. Check the delivered response, not your CMS settings:
# Header form
curl -sI -A "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" \
https://www.example.com/page | grep -i x-robots-tag
# Meta tag form, in the raw HTML
curl -s -A "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" \
https://www.example.com/page | grep -i 'name="bingbot"'Then confirm three things:
- The tag is in the raw HTML, not only added later by JavaScript.
- robots.txt does not block Bingbot from the page, or the tag will never be read.
- No stray
ai-trainingorai-groundingvalue sits on a test page you forgot, since any new preference makes Bing disregardnoarchiveandnocachesite-wide.
Frequently asked questions
Does disallowing AI grounding remove my page from Bing search?
No. Microsoft says the preferences do not change eligibility for traditional Bing results. Use noindex if you want a page out of Bing search.
Can I set AI grounding in robots.txt?
No. Microsoft states there is no site-wide robots.txt grounding preference. Grounding is set per page with the Bingbot meta tag or the X-Robots-Tag header.
What happens if I keep using NoArchive after March 2027?
After 1 March 2027 Microsoft no longer reads noarchive or nocache as training or grounding preferences. Pages relying on them without a recorded temporary preference default to Allow for grounding until recrawled.
What is the difference between disallow and disallowextended?
disallow stops the page informing generated responses at all. disallowextended still allows Microsoft-operated experiences such as Copilot and Microsoft 365, but excludes grounding services offered to third parties, such as Web IQ and Foundry.
Does an allow value let Bingbot crawl pages blocked in robots.txt?
No. Microsoft says an Allow setting does not override a robots.txt rule blocking Bingbot. Usage preferences only apply to pages Bingbot can fetch.
Check that Bingbot can read your preferences
New tags only work on pages Bingbot can reach and parse. Run the free check on a page you have tagged to see whether the main AI crawlers can fetch it. It needs no account.
Filed under Access and robots.txt. Topics: bingbot, meta-robots, ai-training, robots-txt, ai-search