Skip to content

WooCommerce SEO for AI crawlers: product pages, schema, robots.txt

Make WooCommerce product pages readable for AI crawlers: built-in Product schema, robots.txt for cart and filter URLs, and plugin conflicts that block bots.

By Crawlito · 6 min read

In short

  • WooCommerce renders product pages on the server in PHP, so the product name, price and description are in the HTML crawlers receive unless a theme or plugin moves them into JavaScript.
  • WooCommerce core outputs Product JSON-LD with offers, SKU, availability and reviews, but it does not add a brand property, which you can add with the woocommerce_structured_data_product filter.
  • WooCommerce adds a noindex robots meta tag to its cart, checkout and account pages by default.
  • Add-to-cart links, sorting and layered-navigation filter parameters create many crawlable URLs; Google recommends blocking unneeded faceted URLs in robots.txt.
  • Security and caching plugins are the usual reason a WooCommerce store blocks crawlers or serves them stale prices.
On this page

WooCommerce is in a good starting position for AI crawlers: product pages are rendered on the server, so the name, price, description and stock status are in the HTML a crawler receives, and WooCommerce core adds Product structured data as JSON-LD. What usually goes wrong is everything around the product page: crawlers wandering through add-to-cart and filter URLs, a security plugin that blocks bot user agents, or a cache that serves an old price.

This guide covers what WooCommerce does by default, what to add, which URLs to keep crawlers out of, and how to check each part on your own store. For general WordPress settings such as "Discourage search engines", start with our WordPress AI search readiness guide.

Is WooCommerce product content visible to crawlers?

Usually yes. WooCommerce templates are PHP, so a product page's title, price, short description, long description and attributes are printed into the HTML before it leaves the server. A crawler that does not run JavaScript gets the same core text a browser shows.

The exceptions come from what you add on top:

  • Block or page-builder themes that load product tabs, descriptions or reviews after the page loads.
  • Variation prices for variable products. The price range is in the HTML, but the exact price for a chosen variation is updated in the browser when a shopper picks options.
  • Third-party widgets for reviews, size guides, specs or Q&A that fetch content with JavaScript.
  • "Load more" or infinite scroll on category pages, which can hide products beyond the first batch. See pagination and infinite scroll.

To check a product page, fetch it without a browser and look for a sentence from the description and the price:

bash
curl -s -A "GPTBot/1.1" https://shop.example.com/product/blue-mug/ | grep -c "Hand-glazed stoneware"
curl -s -A "GPTBot/1.1" https://shop.example.com/product/blue-mug/ | grep -o 'woocommerce-Price-amount[^<]*<[^>]*>[^<]*' | head -3

If the description count is zero, that content is being added after load and crawlers that skip JavaScript will not see it.

What structured data WooCommerce outputs by default

WooCommerce core has a structured data class, WC_Structured_Data, that collects markup while the page is built and prints it as one JSON-LD block in the footer. On a single product page you typically get:

TypeWhereMain properties
ProductSingle product pagesname, url, description, image, sku (falls back to the product ID), gtin when a valid one is set
Offer or AggregateOfferInside Productprice or lowPrice/highPrice, priceCurrency, priceValidUntil for scheduled sales, availability (InStock, OutOfStock, BackOrder), seller
AggregateRating and ReviewInside Product, when approved reviews existRating value, review count, recent reviews
BreadcrumbListPages that show WooCommerce breadcrumbsBreadcrumb trail
WebSite with SearchActionShop page when it is the front pageSite search

You can read the exact logic in WooCommerce's source on GitHub, which is the most reliable reference because the markup changes between versions.

Two gaps are worth knowing:

  • No brand. Core does not map a brand property. Google lists brand as a recommended property for product markup, so stores often add it.
  • GTIN only when filled in. Recent WooCommerce versions read a global unique ID field on the product (GTIN, UPC, EAN or ISBN) and output gtin only if the value is valid. Empty fields produce no gtin.

Adding brand with a filter

The woocommerce_structured_data_product filter receives the markup array and the product. Extend it rather than replacing it, so price and availability stay in sync with core:

php
add_filter( 'woocommerce_structured_data_product', function ( $markup, $product ) {
    $brand = $product->get_attribute( 'pa_brand' );
    if ( $brand ) {
        $markup['brand'] = array(
            '@type' => 'Brand',
            'name'  => $brand,
        );
    }
    return $markup;
}, 10, 2 );

Adjust pa_brand to the attribute or taxonomy your store uses for brands.

Avoiding duplicate Product markup

SEO plugins and schema plugins often output their own Product markup. Two Product entities for the same item, with different prices or ratings, are confusing for any consumer of the data. Pick one source: either let the plugin replace WooCommerce's markup (most document a setting for this) or turn the plugin's product schema off. Then validate with Google's Rich Results Test. Our Product schema guide explains which properties Google uses and why the marked-up price must match the visible one.

Which WooCommerce URLs to keep crawlers out of

A WooCommerce store generates many URLs that are not useful to crawl:

URL patternWhere it comes fromDefault handling
/cart/, /checkout/, /my-account/WooCommerce pagesWooCommerce adds a noindex robots meta tag
?add-to-cart=123"Add to cart" buttons on archive pagesCrawlable unless you block it
?orderby=priceSorting dropdownCrawlable
?filter_color=blue&query_type_color=orLayered navigation filtersCrawlable
?min_price=10&max_price=50Price filterCrawlable

The add-to-cart links matter most. They appear on every category and shop page, and each request can add an item to a session cart, which costs server work and can create sessions for bots.

Google's guidance on faceted navigation says that if you do not need filtered URLs in search, you should prevent their crawling, and robots.txt is the most direct way. A starting point for a WooCommerce store:

User-agent: *
Disallow: /*?add-to-cart=
Disallow: /*&add-to-cart=
Disallow: /*?orderby=
Disallow: /*?filter_
Disallow: /*&filter_
Disallow: /*?min_price=
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/

Sitemap: https://shop.example.com/wp-sitemap.xml

Notes on this file:

  • Check your permalinks. If your cart page is /basket/ or your store uses a different language, adjust the paths.
  • Keep filters you want found. If a filtered page such as "blue mugs" is a real landing page, give it a clean URL (a category or tag) rather than relying on a filter parameter.
  • Repeat rules in named groups. If you add a User-agent: GPTBot or User-agent: PerplexityBot group, it replaces the * group for that crawler, so copy these lines into it.
  • Robots.txt and noindex interact. A crawler that is disallowed from /cart/ cannot see its noindex tag. For cart and checkout that is fine, because neither page has anything worth indexing.
  • Your sitemap URL may differ. WordPress core serves wp-sitemap.xml; SEO plugins often replace it with their own.

WordPress generates robots.txt virtually, so to edit it you either use your SEO plugin's editor or upload a physical file, which overrides the virtual one. Our guide to faceted navigation crawl traps has more patterns.

Security plugins that block crawlers

WordPress security plugins and host-level firewalls are the most common reason a WooCommerce store is invisible to AI crawlers. Bot-blocking lists, "block fake Googlebots" options, country blocking and rate limits can all catch legitimate crawlers. Store owners tend to tighten these after a card-testing attack on checkout, and the rules end up applying site-wide.

Protect what needs protecting, which is checkout, account and login endpoints, and leave product and category pages reachable. Then test with crawler user agents:

bash
for ua in "GPTBot/1.1" "OAI-SearchBot/1.0" "PerplexityBot/1.0" "ClaudeBot/1.0"; do
  printf "%-18s " "${ua%%/*}"
  curl -s -o /dev/null -w "%{http_code}\n" -A "$ua" https://shop.example.com/product/blue-mug/
done

Anything other than 200 needs explaining. Our guide to WordPress security plugins that block crawlers walks through the common settings.

Caching without serving stale prices

Page caching makes WooCommerce fast for crawlers and shoppers, but it must leave the dynamic parts alone. WooCommerce's caching guidance says to exclude the Cart, My Account and Checkout pages from caching, and lists cookies you can exclude, including woocommerce_cart_hash, woocommerce_items_in_cart and wp_woocommerce_session_.

For crawlers, the bigger risk is the opposite problem: product pages cached for so long that a price or stock change does not reach the HTML. If a sale ends but the cached page still shows the sale price, the visible price and the JSON-LD price are both stale, and anything a crawler stores is out of date. Make sure your cache purges a product page when the product is saved, including scheduled sale start and end times. See CDN cache and stale content for how to check.

Frequently asked questions

Does WooCommerce add Product schema automatically?

Yes. WooCommerce core outputs Product JSON-LD on single product pages, including offers, SKU, availability and reviews where they exist. It does not include brand, which you can add with the woocommerce_structured_data_product filter.

Should I block add-to-cart URLs in robots.txt?

Usually yes. They are action URLs, not content, and blocking them with Disallow: /*?add-to-cart= stops crawlers from triggering cart operations while browsing category pages.

Does WooCommerce noindex the cart and checkout pages?

Yes. WooCommerce adds a noindex robots meta tag to its transactional pages such as cart, checkout and my account.

Are variable product prices visible to crawlers?

The price range is in the server HTML and in the AggregateOffer markup. The exact price for a selected variation is updated in the browser, so crawlers see the range, not each combination.

Check a product page

The quickest way to know what crawlers get from your store is to test a real product URL. Run the free check on one, or book an audit to review product pages, robots.txt and plugin settings across the store.

Filed under Platform guides. Topics: wordpress, ecommerce, structured-data, robots-txt, ai-crawlers