WordPress.com AI crawlers: what third-party sharing blocks
WordPress.com's 'Prevent third-party sharing' option adds known AI bots to robots.txt. What it covers, what WordPress.com does not document, and how to check.
By Crawlito · 8 min read
In short
- WordPress.com says 'Prevent third-party sharing' does two things: it excludes your public content from its network of third-party content and research partners, and it adds known AI bots to the disallow list in your robots.txt.
- WordPress.com's support page does not list which AI bots the option adds, so whether a specific crawler such as OAI-SearchBot is included is not documented; only your live robots.txt shows it.
- WordPress.com says it is up to AI platforms to honor the robots.txt request, and RFC 9309 says robots.txt rules are not a form of access authorization.
- 'Discourage search engines from indexing this site' is a separate checkbox aimed at search engines, and WordPress.com warns that not all search engines may respect it.
- Both options sit under Settings > Reading > Site Visibility, are ticked after you select Public, and WordPress.com says the sharing opt-out must be set on each site you manage.
On this page
- What does "Prevent third-party sharing" do on WordPress.com?
- Where do you turn it on?
- What does it change in robots.txt, and which bots does it name?
- How does it differ from "Discourage search engines"?
- Why are AI search crawlers a separate question from partner sharing?
- How do you check your robots.txt from outside?
- WordPress.com vs self-hosted WordPress: which controls do you have?
- Frequently asked questions
- Check what AI crawlers can read on your site
On WordPress.com, the option to use is Prevent third-party sharing, under Settings > Reading > Site Visibility. WordPress.com says it keeps your public content out of its network of third-party content and research partners and adds known AI bots to the disallow list in your robots.txt file. It does not publish which bots are on that list, and it says AI platforms decide whether to honor the request. So turn it on if you want to opt out, then read your live robots.txt to see what it actually says.
Facts about WordPress.com in this guide come from its support pages, read on 11 November 2026. Settings and wording can change, so check your own dashboard.
What does "Prevent third-party sharing" do on WordPress.com?
WordPress.com's guide to making your site public describes two effects.
- Partner sharing. The option removes your site's public content from WordPress.com's network of third-party content and research partners. WordPress.com's privacy settings page describes that network as a licensed network of content and research partners, including those that train AI models.
- robots.txt. The option adds known AI bots to the disallow list in your site's robots.txt file.
The guide adds a caveat: AI platforms are not obliged to honor the robots.txt request. That matches how the standard works. RFC 9309 says robots.txt rules are not a form of access authorization, so the file is a request to crawlers that choose to follow it. Our post on whether AI crawlers respect robots.txt covers what operators document and what research has found.
Notice that the two effects work differently. The first concerns WordPress.com's own licensed partner network. The second is a line in your robots.txt that outside crawlers can read. They reach different audiences.
Where do you turn it on?
The steps from WordPress.com's guide:
- Open your site's dashboard.
- Go to Settings > Reading.
- Scroll to Site Visibility.
- Select Public, then tick Prevent third-party sharing.
- Click Save Changes.
Three details are easy to miss:
- You select Public first. The guide's steps say to select Public and then tick the box. It also says that if you do not see Public, the site has not been launched yet, so launch it first.
- It is per site. If you manage several sites, WordPress.com says you must set the option on each one.
- Coming soon and Private are different states. The privacy settings page describes Coming soon as hidden from visitors while you work on it, and Private as visible only to you and users you approve. This guide is about public sites that want to opt out of AI use.
What does it change in robots.txt, and which bots does it name?
This is the part WordPress.com does not spell out. The support page says "known AI bots" are added to the disallow list. It names no bots, gives no list URL and does not say how often the list is updated.
That leaves three questions the documentation does not answer:
| Question | What WordPress.com documents |
|---|---|
| Which bots are added? | Not documented; the page says "known AI bots" |
| Are AI search crawlers such as OAI-SearchBot included, or only training crawlers? | Not documented |
| Does the list update when new AI bots appear? | Not documented |
The last two matter because training and search are separate jobs. OpenAI's crawler documentation says sites that opt out of OAI-SearchBot are not shown in ChatGPT search answers, though they can still appear as navigational links. It also says GPTBot and OAI-SearchBot settings are independent: blocking GPTBot signals that content should not be used for training, but does not touch search. If your goal is "no training, but keep appearing in AI search", you need to know which side of that line the added bots fall on. AI training crawlers vs search crawlers explains the split in full.
Do not assume the answer either way. Read the file.
How does it differ from "Discourage search engines"?
They sit side by side as checkboxes, but they aim at different crawlers.
| Prevent third-party sharing | Discourage search engines from indexing this site | |
|---|---|---|
| Aimed at | AI platforms and partner networks | Search engines |
| Documented effect | Excludes content from partner network; adds known AI bots to robots.txt disallow | Blocks most search engine crawlers, per the support page |
| Honored by everyone? | No: AI platforms decide | No: WordPress.com says not all search engines may respect it |
| Other side effects | None stated beyond the two above | Excludes your posts from the WordPress.com Reader |
Two more points:
- The WordPress.com page does not say whether "Discourage search engines" works through a robots.txt rule or a noindex robots meta tag on WordPress.com. On self-hosted WordPress, the Reading settings documentation says the box adds a
noindex,nofollowmeta tag since version 5.3, and that it is up to search engines to honor the request. Do not assume the two platforms behave the same. - "Discourage search engines" is aimed at search engines, so ticking it to stop AI asks search engines to stay away from a site you may want them to list. If your aim is to opt out of AI use, "Prevent third-party sharing" is the option the documentation ties to AI. The guide presents them as separate checkboxes.
Why are AI search crawlers a separate question from partner sharing?
Partner sharing and crawling are two different routes by which content can reach an AI system.
- Crawling happens from the outside. A crawler such as GPTBot or OAI-SearchBot fetches your public pages, reads robots.txt first if it follows the standard, and takes whatever your server returns. Your robots.txt is the lever you control.
- Partner licensing concerns WordPress.com's own network of content and research partners. The support page describes the opt-out as excluding your content from that network, not as a request your site answers.
So one checkbox covers both routes, but only the crawling route is something you can check from outside. You can verify a robots.txt line. You cannot verify, from the outside, what WordPress.com does or does not share with partners. Treat that part as a statement from WordPress.com rather than something you tested.
Also remember the user-triggered category. OpenAI says robots.txt rules may not apply to ChatGPT-User, which visits a page when a person asks ChatGPT about it. A robots.txt disallow is not a reliable switch for that kind of visit, whatever the platform adds to your file.
How do you check your robots.txt from outside?
Fetch the file the way a crawler does, from your public address:
curl -s https://your-site.example/robots.txtReplace the address with the one visitors use, which may be a custom domain or your .wordpress.com address. Then read the output:
- Look for named AI groups. Search for
User-agent:lines that name AI crawlers, for exampleGPTBot,OAI-SearchBot,ClaudeBot,Claude-SearchBotorPerplexityBot. - Read the rule under each.
Disallow: /under a named group blocks the whole site for that crawler.Allow: /under a named group allows it. If no group names the crawler, it falls back to the*group, if there is one. - Compare before and after. Save the file, tick the option, save the setting, fetch it again, and diff the two. The difference is what the option added on your site today.
curl -s https://your-site.example/robots.txt > before.txt
# tick Prevent third-party sharing, click Save Changes, then:
curl -s https://your-site.example/robots.txt > after.txt
diff before.txt after.txtIf the file does not change, open the Reading screen again to confirm the box is ticked and saved, then fetch the file again. WordPress.com does not document how quickly the file updates, so a diff that shows no change is a reason to look closer, not proof of either outcome.
Remember how groups work. A crawler obeys the group that names it and ignores the * group, as robots.txt for AI crawlers explains. If you want to test one named crawler against the rules, how to test robots.txt against each AI crawler shows several ways. And allow time for changes to land: OpenAI says its search systems can take about 24 hours to adjust after a robots.txt update, and other operators do not publish a schedule.
WordPress.com vs self-hosted WordPress: which controls do you have?
On self-hosted WordPress you control the server, so you control robots.txt directly. On WordPress.com the support pages I read describe the checkbox and not a way to write your own AI rules. I did not find, in those pages, a statement of whether or on which plans you can edit robots.txt, so check your own plan's options rather than relying on a third-party guide.
| WordPress.com | Self-hosted WordPress | |
|---|---|---|
| AI opt-out control | "Prevent third-party sharing" checkbox | Your own robots.txt, a plugin or a firewall rule |
| Which bots are named | Not documented | Whatever you write |
| Partner sharing | WordPress.com's network, opt out with the checkbox | Not applicable |
| Server-level blocking | Not described in the support pages | Possible with a firewall or server config |
If you run self-hosted WordPress, the checkbox above does not exist. Use WordPress AI crawlers: the settings that decide what they see and how to allow AI search but block AI training instead.
Frequently asked questions
Does "Prevent third-party sharing" block ChatGPT search?
WordPress.com does not say. It says the option adds "known AI bots" to the robots.txt disallow list without naming them. OpenAI says sites that opt out of OAI-SearchBot are not shown in ChatGPT search answers, so read your robots.txt to see whether that crawler is named.
Is the option enough to stop every AI crawler?
No. WordPress.com says it is up to AI platforms to honor the robots.txt request, and RFC 9309 says robots.txt rules are not access authorization. A crawler that ignores the file can still fetch your public pages, and OpenAI says robots.txt rules may not apply to ChatGPT-User.
Is "Discourage search engines" the same as blocking AI?
No. It is a separate checkbox aimed at search engines, and WordPress.com warns that not all of them respect it. It also excludes your posts from the WordPress.com Reader. For AI, the documented option is "Prevent third-party sharing".
Do I need to repeat the setting for every site?
Yes. WordPress.com says that if you manage multiple sites, you must set the option on each one.
Check what AI crawlers can read on your site
The free check reads a single page the way AI crawlers do, with no account, and shows what your robots.txt says to each named AI crawler. For a whole site, the audit covers up to 500 pages with evidence and prioritized fixes.
Filed under Platform guides. Topics: wordpress, ai-crawlers, robots-txt, privacy-settings