Skip to content

Zendesk help center noindex: what you can control for crawlers

Zendesk help center noindex explained: the document_head.hbs tag, restricting articles to signed-in users, robots.txt, sitemaps and how to test what crawlers see.

By Crawlito · 9 min read

In short

  • Zendesk documents one way to noindex a public help center: add <meta name="robots" content="noindex" /> to the theme's document_head.hbs template and publish the theme.
  • Zendesk says that tag prevents search engines from indexing all pages in the theme; it documents no theme method for noindexing a single article.
  • To keep individual articles out, Zendesk says to restrict them to signed-in users with user segments; search engines do not index articles that require a login.
  • Zendesk's help center documentation describes an automatic sitemap at /hc/sitemap.xml, referenced from the help center's robots.txt, but documents no way to edit robots.txt at the time of writing.
  • Google needs to recrawl a page to see noindex, and Zendesk warns that removal from the index can take months.
On this page

Zendesk gives you two documented controls over how search engines treat a Zendesk Guide help center. To keep a public help center out of search results, you add <meta name="robots" content="noindex" /> to the theme's document_head.hbs template, which Zendesk says applies to all pages in the theme. To keep single articles out, you restrict them to signed-in users with user segments, because search engines cannot index pages that require a login. Zendesk documents no way to edit the help center's robots.txt, so robots.txt is not one of your levers.

This post covers those two controls, what they do to Google and to AI crawlers, how host mapping and the sitemap fit in, and how to test what a crawler actually receives. If you are deciding which support content should be public in the first place, start with knowledge bases for AI search; this post assumes that decision is made and deals with the Zendesk mechanics.

How to noindex a whole Zendesk help center

Zendesk's article Can I restrict Google or other search engines from indexing my help center? gives the procedure for a public help center:

  1. In Knowledge admin, click the Customize design icon in the sidebar.
  2. Click your theme to open it.
  3. Click Edit code and select document_head.hbs.
  4. Add the robots meta tag.
  5. Click Publish.

The line to add:

html
<meta name="robots" content="noindex" />

Two things follow from where this tag lives.

It is site-wide. Zendesk says this code "prevents search engines from indexing all pages in the theme", not one article. It does not document a way to noindex a single article through the theme, so treat any conditional theme code you write for that as your own change, and test the pages it should and should not affect.

It belongs to the theme, not the account. If you switch to another theme, or a colleague publishes a new version of the theme without the line, the tag is gone. Keep it in whatever version control or change notes you use for theme code, and check it after every theme publish.

The generic robots name matters. Google's robots meta tag documentation says it "specifies that the rule applies to all crawlers"; replacing robots with a crawler's name, such as googlebot, addresses only that crawler. Whether a given AI crawler honours noindex is up to its operator, and the operators document this very differently: see noindex and nosnippet for AI search for what each one says.

For individual articles, Zendesk's answer is access control, not a meta tag. The same Zendesk article says to use user segments to limit specific articles to signed-in users, and that search engines don't index articles that require a login.

Zendesk's article view permissions page sets this per article, under Article settings > Placement > Viewing permissions:

Viewing permissionWho can read itCan an anonymous crawler read it?
Visible to everyoneAnyone, no sign-inYes, if nothing else blocks it
Only visible to selected user segments: Signed-in usersAnyone with an account who signs inNo
Only visible to selected user segments: Agents and adminsTeam members onlyNo
Only visible to selected user segments: Custom user segmentUsers matched by tags, organizations or groupsNo

A few documented details are worth knowing before you change many articles:

  • Permissions are set at the article level, not on sections or categories. To change a whole section, Zendesk suggests filtering an article list by section and bulk updating it.
  • If every article in a section is internal, the section itself is hidden from end users. One article visible to end users makes the section visible again.
  • On Suite Team, article permissions are limited to everyone or signed-in users; agents-and-admins and custom segments are not available there.

The trade-off is plain: a restricted article is out of search because nobody without an account can read it, including customers who arrive from a link. That is the right tool for internal or customer-specific material. It is the wrong tool for a public article you only want out of Google, and for that case Zendesk documents nothing narrower than the site-wide tag. Our login walls and AI crawlers guide covers what gated content means for AI search tools more broadly.

Restricting the whole help center

Zendesk's restricted help center article describes Require sign in under Knowledge admin > Settings > Security. Anonymous visitors then see only the sign-in page. Zendesk's noindex article adds that a help center which requires sign-in is not indexed by search engines by default, and that the noindex tag is needed only when anyone can view the help center without a login. Employee Service plans are always restricted to signed-in users.

What Zendesk says about robots.txt

At the time of writing, Zendesk's help center documentation has no article on editing a help center's robots.txt file. Searching Zendesk's own help center for "robots.txt" returns articles about its own web crawler for external content, the SEO overview and the noindex article above, but no editing procedure. Assume you cannot add rules for GPTBot, ClaudeBot or any other user agent to the help center's robots.txt.

What Zendesk does document is that the file exists and points to the sitemap. Its help center SEO article says the sitemap "is accessible to search engine crawlers via the robots.txt file on your help center". Zendesk's own help center shows what that looks like. An excerpt of the file served at support.zendesk.com/robots.txt, read on 15 November 2026:

# Excerpt: the full file has more Disallow lines and a separate AppleBot group
User-agent: *
Disallow: /hc/*/signin
Disallow: /hc/requests/
Disallow: /hc/*/requests/
Allow: /hc/*/requests/new
Disallow: /hc/*/search
Disallow: /access/
Disallow: /auth/

Sitemap: https://support.zendesk.com/hc/sitemap.xml

That is one help center's file, and Zendesk does not document whether every help center receives the same rules. Fetch your own to see what applies to you.

This has a practical consequence for noindex. robots.txt controls fetching, and meta robots controls indexing: Google's block indexing page says a page blocked by robots.txt is never fetched, so its noindex is never seen. Article URLs under /hc/<locale>/articles/ are not disallowed in the file above, so a crawler can read the tag. The robots.txt vs meta robots guide explains the interaction in detail.

If you need crawler-specific blocking, for example keeping training crawlers out while allowing search crawlers, the help center itself does not give you that control. It has to happen at a layer you run, such as a CDN or proxy in front of a host-mapped domain, and Zendesk's host mapping instructions for Cloudflare DNS ask for the record to be set to DNS only, not proxied. Check Zendesk's current guidance before putting a proxy in front.

Host mapping and the sitemap

Host mapping changes the external address of a help center from yoursubdomain.zendesk.com to a subdomain of your own, such as support.example.com, through a CNAME record and a hosted SSL certificate. Zendesk notes it works only for subdomains, not for a subdirectory such as example.com/help, and that it does not change the sign-in page or the agent interface URLs. The host mapping article warns that without a Zendesk-provisioned or uploaded certificate, traffic to the mapped domain gets SSL errors, and a separate troubleshooting article ties redirects to the zendesk.com address to hosted SSL not being set up.

For crawlers, that means:

  • The help center lives on its own host. Its robots.txt is the one at support.example.com/robots.txt. Rules in your main site's robots.txt do not apply to it.
  • Check which host your pages declare. Fetch an article on the mapped domain and on the zendesk.com address, and compare the final URL and the canonical tag. You want one host to be the canonical address.

The sitemap, per Zendesk's SEO article:

FactWhat Zendesk documents
Locationsubdomain.zendesk.com/hc/sitemap.xml
ContentKnowledge base categories, sections and articles; community content is not included
UpdatesAutomatic, every two hours
Restricted help centersNo sitemap is generated when sign-in is required
PlansNot available on Suite Team

On Zendesk's own help center, /hc/sitemap.xml is a sitemap index pointing to numbered files under /hc/sitemap/, with hreflang alternates for each locale. Submit the sitemap at the host you want indexed in Google Search Console and Bing Webmaster Tools. Zendesk does not say the sitemap changes when you add the site-wide noindex, so if you do, expect Search Console's page indexing report to list sitemap URLs as excluded by a noindex tag.

How to test what a crawler gets from a Zendesk article

Test from outside your network, signed out, against a public article URL. A request with a crawler's user agent shows what the help center returns to that user agent; it does not prove what a verified crawler from that operator receives, but it catches most problems.

bash
URL="https://support.example.com/hc/en-us/articles/123456789-Example"

UA="Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot"

# Status, redirects and any X-Robots-Tag header (a GET request, headers only)
curl -sL -o /dev/null -D - -A "$UA" "$URL"

# Robots meta tag and canonical in the HTML
curl -sL -A "$UA" "$URL" \
  | grep -oiE '<meta name="robots"[^>]*>|<link rel="canonical"[^>]*>'

Use a GET request rather than curl -I. On 15 November 2026, a HEAD request for an article on Zendesk's own help center returned 404, while a GET for the same URL returned 301 to the slugged article URL and then 200, so a HEAD test can make a public article look missing.

What to read in the output:

ResultMeaning
200 and the article text is in the HTMLReadable; check for a robots meta tag next
<meta name="robots" content="noindex" /> presentYour theme tag is live on this page
302 to an /access or sign-in URLThe article or help center requires sign-in
403 or a challenge pageSomething in front of the help center is blocking the user agent

As a reference point, a public article on Zendesk's own help center returned 301 to its slugged URL and then 200 with the full article text in the raw HTML and a canonical tag when fetched with the GPTBot and OAI-SearchBot user agents on 15 November 2026, while its /hc/en-us/requests page returned 302 to an /access URL. Repeat the test with the user agents you care about; see your page as an AI crawler with curl lists them and covers the edge cases.

For Google specifically, use the URL Inspection tool in Search Console, which shows the HTML Googlebot received and whether it found a noindex rule. Removal takes a recrawl: Google says this can take months for less important pages, and Zendesk gives the same warning.

Frequently asked questions

Can I noindex a single Zendesk article with a meta tag?

Zendesk does not document a per-article robots meta tag. Its documented method for single articles is to restrict them to signed-in users with user segments, which also hides them from anonymous readers.

Can I edit robots.txt on a Zendesk help center?

Zendesk's documentation describes no way to edit it at the time of writing. Zendesk documents only that the file exists and points crawlers to the help center sitemap.

Does the noindex tag stop AI crawlers from reading my help center?

No. noindex controls indexing, not fetching, so crawlers can still request the pages. Whether an AI crawler respects the tag depends on its operator; only requiring sign-in stops every crawler from reading the content.

Does a noindexed help center still appear in Google AI Overviews?

Google says a page must be indexed and eligible to be shown with a snippet to appear as a supporting link in AI Overviews or AI Mode. A page with noindex is not indexed, so it is not eligible.

Is the Zendesk sitemap created for restricted help centers?

No. Zendesk says the sitemap is not generated when the help center requires sign-in, and web crawlers cannot access closed or restricted help centers.

Check one of your help center articles

After changing theme code or article permissions, test a public article and a restricted one to confirm each gets the response you intended. The free check tests a single page, with no account, to show whether the main AI crawlers can fetch it.

Filed under Platform guides. Topics: noindex, meta-robots, documentation, zendesk