Skip to content

A/B testing and SEO: what crawlers see in tests and personalization

How A/B tests, geo and cookie personalization affect Google and AI crawlers: Google's testing rules, empty variants in raw HTML, and how to check what bots get.

By Crawlito · Updated 7 min read

In short

  • Google's testing guidance says not to cloak, to use rel=canonical on variant URLs, to redirect with 302 rather than 301, and to run a test only as long as necessary.
  • Google says Googlebot generally does not support cookies, so cookie-based tests and personalization usually show crawlers one default version.
  • Googlebot's default crawling IPs appear to be in the US, and it sends no Accept-Language header, so geo-personalized pages can show it content most of your visitors never see.
  • Client-side testing tools that hide the page until a variant loads can leave a non-rendering AI crawler with blank or placeholder HTML.
  • The test for all of these is the same: compare the raw HTML a crawler gets with what a typical visitor sees, for every variant and region.
On this page

A/B tests and personalization are compatible with search, as long as every crawler gets a real, complete version of the page and that version matches what people see. Google's rules for testing are short: do not show crawlers different content on purpose, mark variant URLs with rel="canonical", use 302 redirects for tests, and end the test when you have your answer. The bigger practical risk is technical: client-side test scripts and geo or cookie rules can leave crawlers, especially AI crawlers that do not run JavaScript, with an empty or unusual version of the page.

This post covers what Google documents, what goes wrong with each kind of test, and how to check what crawlers actually receive.

What Google says about A/B testing

Google's page on minimizing A/B testing impact in Google Search gives four rules:

  1. Don't cloak your test pages. Do not show one set of URLs to Googlebot and a different set to humans. Google says cloaking is against its spam policies whether or not you are running a test.
  2. Use rel="canonical" links. If a test uses separate URLs for variants, put a canonical link on every variant URL pointing to the original. Google prefers this to noindex.
  3. Use 302 redirects, not 301. A temporary redirect tells Google the original URL should stay indexed. Google also accepts JavaScript-based redirects for tests.
  4. Run the experiment only as long as necessary. Once you have reliable data, remove the test code. Google says a long-running test, especially one where most users see one variant, can be read as an attempt to deceive.

The same page notes that you can run tests without changing URLs by using JavaScript to decide which variation to display, and that Googlebot generally does not support cookies, so it may see only one version of a cookie-based test.

The wider question of serving different content to bots is covered in our post on serving AI crawlers different content.

The four kinds of test and what crawlers get

Test typeHow it worksWhat a crawler usually getsMain risk
Split URL (redirect) testSome visitors are redirected to /page-bWhichever URL it requests, plus any redirect your rule sends itPermanent redirects or missing canonicals on variants
Server-side testServer picks a variant, same URLOne variant, chosen by your bucketing ruleBucketing that treats bots differently on purpose
Client-side testScript swaps content after loadRaw HTML before the script, or a hidden pageBlank or placeholder content in raw HTML
PersonalizationContent varies by location, cookie or loginThe default version for a cookieless US visitor, typicallyDefault version is thin or wrong

Split URL tests

Check three things on every variant URL:

  • It returns 200 and links to the original with <link rel="canonical" href="https://www.example.com/page">.
  • The redirect from the original is a 302 or 307, never 301 or 308.
  • Variant URLs are not in your sitemap and are not linked from navigation.
bash
curl -sI https://www.example.com/page | grep -iE "^(HTTP|location)"
curl -s https://www.example.com/page-b | grep -i 'rel="canonical"'

Our guide to canonical tags and duplicate URLs explains how Google treats the canonical as a hint, which is why the other two checks matter too.

Server-side tests

Server-side tests are the cleanest for crawlers, because every response is complete HTML. The decision to get right is how bots are bucketed. Sending all bots to the control version is common and generally harmless when the control is what most visitors see. What crosses the line is building a variant for bots that visitors never get, such as a keyword-heavy version.

If your testing framework buckets by cookie, a crawler without cookies gets bucketed fresh on every request and can see different variants from one fetch to the next. That is allowed, but it can make the page look unstable. Bucketing cookieless requests to the control is a reasonable default.

Client-side tests

This is where most problems show up. Many client-side testing tools hide the page, or part of it, until the testing script has decided which variant to show. That prevents visitors from seeing the original flash before the change, often called flicker. It also means:

  • A crawler that does not run JavaScript gets the raw HTML. If the test replaces a headline, it sees the original headline, which is fine. If the test inserts a whole section, such as new pricing copy, that section is not in the raw HTML at all.
  • If the anti-flicker rule hides the body with CSS and relies on the script to reveal it, the raw HTML still contains your text, but a renderer whose script fails or times out can capture a blank page.
  • Some setups ship a placeholder in the HTML, such as an empty <div id="hero"></div>, and fill it only from the test script. Every non-rendering crawler then sees an empty hero for the duration of the test.

The rule of thumb: the control version should be fully present in the server HTML, and a test should change it, not create it. See raw vs rendered HTML for the general way to compare the two.

Personalization is not a test, but it creates the same question: which version does a crawler get?

Google's guidance on locale-adaptive pages says Googlebot's default IP addresses appear to be in the USA, and that it sends requests without an Accept-Language header. Google also crawls from some non-US addresses, but does not say how often, so treat the US version as the default it sees. Google recommends separate locale URLs annotated with hreflang over adapting one URL by location. Our post on hreflang for AI search covers that setup.

For AI crawlers, operators do not document where every fetch comes from, so treat location as unknown. Practical consequences:

  • A geo redirect can send crawlers away from the page they asked for. If a visitor from the US requesting /de/ is redirected to /us/, a crawler from a US address can never fetch the German page. Show a banner suggesting the local version instead of redirecting.
  • Region-specific blocks hide content entirely. If a page or a store is blocked in some regions, a crawler from those regions gets the block. The post on geo-blocking and IP allowlists covers how to check this.
  • Cookie-based personalization defaults matter. Crawlers generally arrive without cookies. Whatever you show a first-time, cookieless visitor is what crawlers record. If that version is a thin "choose your region" page, that is your indexed page.
  • Consent banners count too. A cookie wall that replaces content until consent is given can be all a crawler sees. The guide to cookie walls and interstitials shows what crawlers get from each pattern.

How to check what crawlers see during a test

Run this for every live test and for each personalized template.

  1. Fetch the raw HTML without cookies, the way a crawler does:

    bash
    curl -s -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
      https://www.example.com/page > crawler.html
  2. Search it for the content each variant should have. A sentence from the control and a sentence from the test variant:

    bash
    grep -c "Plans from \$29" crawler.html
    grep -c "Start free for 14 days" crawler.html

    The control sentence should be present. If neither is, the content depends on the test script.

  3. Look for hiding rules such as opacity: 0 or visibility: hidden on body or html, and for empty containers the test fills.

  4. Repeat the request several times. If the result changes between requests, crawlers are being bucketed into different variants.

  5. Check the rendered version in Google's URL Inspection tool to confirm the renderer ends up with visible content.

  6. For geo rules, request from another region through a VPN or a remote machine, and compare. Note any redirect in the response headers.

The user-agent in step 1 tests rules keyed on the user-agent string only. It does not reproduce the IP address or the network a real crawler uses.

A checklist for running tests safely

  • The control version is complete in the server HTML.
  • Variant URLs have canonicals to the original and are reached by 302 or 307.
  • Bots are not given content that no visitor sees.
  • Anti-flicker hiding has a short timeout and does not apply when the script fails to load.
  • Cookieless, first-time visitors get a full page, not a chooser or wall.
  • Geo logic suggests rather than forces a redirect.
  • Tests end and their code is removed when you have the result.

Frequently asked questions

Can A/B testing hurt SEO?

Google publishes rules for testing in a way that minimizes the effect on Search: no cloaking, canonicals on variant URLs, temporary redirects and a limited duration. Problems usually come from breaking those rules or from test code that leaves crawlers with incomplete HTML.

Should I exclude bots from A/B tests?

Sending bots to the control version is a common, reasonable choice, provided the control is what most visitors see. Do not create a special version for bots.

Why does Google index the wrong version of my personalized page?

Googlebot generally does not keep cookies, sends no Accept-Language header, and its default crawling IPs appear to be in the US (it also crawls from some non-US addresses). So it usually sees your default, cookieless, US version. Use separate URLs for languages and regions if the versions differ.

Do AI crawlers see client-side A/B test variants?

Crawlers that do not run JavaScript see only the server HTML, so they see the original, not the variant. If the original is a placeholder filled by the test script, they see the placeholder.

Check a page under test

Pick a page with an active test or personalization rule and run the free check on it. It shows whether the text visitors read is in the HTML a crawler receives, with no account needed.

Filed under Rendering and content. Topics: rendering, javascript, google, ai-crawlers, canonical