Lazy loaded content and crawlers: tabs, accordions and scroll

Lazy loaded content is readable by crawlers only when its text is already in the HTML the server sends. A closed accordion built with <details>, or a tab panel hidden with CSS, is fine: the words are in the page, just not on screen. Content that is fetched after a click, a scroll or typing is a different matter. Crawlers don't click, and most AI crawlers don't run the JavaScript that would fetch it, so to them that content does not exist.

The rule of thumb is simple: hiding is fine, fetching is not. The rest of this post shows where common patterns fall, how to test your own pages in a few minutes, and what to change when a pattern fails.

Why interaction-based loading fails for crawlers

A crawler requests a URL, reads the response and moves on. It does not open menus, press tabs or scroll to the bottom of a feed.

Google says this plainly. Its lazy-loading guidance states that Google Search does not interact with your page, and its mobile-first indexing best practices say: "Don't lazy-load primary content upon user interaction." Googlebot does render JavaScript with an evergreen Chromium, but rendering a page is not the same as using it.

AI crawlers usually see even less. In Vercel and MERJ's December 2024 analysis, none of the major AI crawlers rendered JavaScript at the time, including OpenAI's crawlers, ClaudeBot and PerplexityBot. ChatGPT's and Claude's crawlers downloaded JavaScript files but did not execute them. That is a snapshot, not a permanent rule, but it means you should plan for crawlers that read the raw HTML and nothing else. For the wider picture, see Do AI crawlers run JavaScript?.

So there are two separate questions for any hidden content:

1. Is the text in the raw HTML response? If yes, non-rendering AI crawlers can read it.
2. Does it load without any user action? If yes, Googlebot can read it after rendering.

Content that fails the first question but passes the second is visible to Google and invisible to most AI crawlers. Content that fails both is invisible to everyone.

Which patterns are readable: a comparison

The middle column is the one most sites get wrong, because the page looks perfect in a browser.

Tabs and accordions: when hidden content is fine

Hiding text to save space is a normal design choice, and Google's mobile-first indexing page even suggests it: "Instead of removing content, consider moving content into accordions or tabs to save space." What matters is how the hiding is done.

The most robust accordion is plain HTML. The browser handles opening and closing, and the answer is in the markup from the start:

<details>
  <summary>Do you ship to Canada?</summary>
  <p>Yes. Orders to Canada ship in 3 to 5 business days.</p>
</details>

Tabs work the same way when every panel is in the HTML and only the visible state changes:

<div role="tablist">
  <button role="tab" aria-selected="true" aria-controls="specs">Specs</button>
  <button role="tab" aria-selected="false" aria-controls="reviews">Reviews</button>
</div>
<section id="specs" role="tabpanel">Weight 1.2 kg. Waterproof to 10,000 mm.</section>
<section id="reviews" role="tabpanel" hidden>4.6 out of 5 from 212 reviews.</section>

A crawler that extracts text from this response gets both panels. How any particular AI system weighs text that is hidden by default is not documented by OpenAI, Anthropic or Perplexity, so don't put the one fact you most want quoted in a collapsed panel if it could just as well sit in the visible copy.

The hidden="until-found" attribute is a useful middle ground. The content stays collapsed but is still in the HTML, and browsers that support it let the reader's find-in-page and fragment links reveal it.

The pattern that breaks

The failure looks like this: the HTML ships an empty panel, and a click handler fills it.

tab.addEventListener("click", async () => {
  const res = await fetch(`/api/product/42/tab/${tab.dataset.id}`);
  panel.innerHTML = await res.text();
});

No crawler clicks that tab, so the reviews, specs or FAQ answers behind it are never read. The same thing happens with some component libraries and framework patterns that only render the active panel and mount the others when they are opened. The page source looks tidy, and the content simply isn't there. Check your own components rather than assuming either way.

The fix: render all panels on the server and toggle visibility on the client. If a panel is genuinely too heavy to include, give that content its own URL and link to it with a normal <a href>, so a crawler can fetch it as a page.

Lazy-loaded images: native loading is fine

The browser's built-in lazy loading is safe for crawlers because nothing about the markup changes. The real image URL and its alt text are in the HTML whether or not the image has been downloaded yet:

<!-- Readable: the real URL and the alt text are in the HTML -->
<img src="/img/boots.jpg" alt="Brown leather hiking boots" loading="lazy" width="800" height="600">

<!-- Risky: the real URL only exists in a data attribute until a script runs -->
<img src="/img/placeholder.gif" data-src="/img/boots.jpg" class="lazyload" alt="">

The second pattern was common before browsers supported loading="lazy". A crawler that reads raw HTML sees a placeholder and an empty alt. Google's own test for lazy-loaded images is to confirm the real URLs appear in the src attribute of the <img> element after rendering.

Two more points from Google's guidance: don't lazy-load content that is likely to be visible as soon as the page opens, and use a method that doesn't depend on user actions such as scrolling or clicking. Google lists the browser's built-in lazy loading, the IntersectionObserver API (with a polyfill if needed) and JavaScript libraries that load data when it enters the viewport.

Infinite scroll needs paginated URLs

Infinite scroll appends the next batch of products or posts when the reader nears the bottom. A crawler gets the first batch and nothing else, and every item after that has no address it could fetch.

Google's guidance is to give each chunk its own persistent URL, use absolute page numbers such as ?page=12 rather than relative values like ?date=yesterday, link to those pages in sequence, and update the browser URL with the History API as a new chunk becomes the main content. The simplest working version is progressive enhancement: ship a real link, then let JavaScript turn it into a "load more" button for people.

<ul id="posts">
  <!-- first 20 posts rendered on the server -->
</ul>
<a href="/blog?page=2" id="load-more">Older posts</a>

Each ?page=N URL should return its own batch in the raw HTML, keep the same content on every load, and link to the next page. Listing the item URLs in your sitemap and linking them from category pages also helps, as covered in Internal links and broken links.

How to test whether hidden content is readable

The test takes five minutes per template. Pick a sentence that only appears inside a collapsed panel, a later tab, or below the first scroll batch.

1. Fetch the raw HTML and search for it. This is what a non-rendering AI crawler receives.

   bash

   curl -s -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
     https://example.com/product/42 | grep -c "212 reviews"

   A count of 0 means the text is not in the response. The step-by-step version, including other user agents, is in How to see your page as an AI crawler, with curl.

2. Watch the network when you open the panel. In Chrome DevTools, open the Network panel, filter by Fetch/XHR, then click the tab or expand the accordion. A new request that returns the panel's text means the content is fetched on interaction.

3. Turn JavaScript off. In DevTools, open the command menu and run "Disable JavaScript", then reload. Whatever is missing now depends on scripts, and you can compare the two versions as described in Raw HTML vs rendered HTML.

4. Check images. Search the raw HTML for your real image file names. If they only appear in data-src or data-srcset, crawlers that don't run scripts see placeholders.

5. Check Google separately. Use URL Inspection in Search Console and look at the rendered HTML for the same sentence. Passing here only tells you about Googlebot, not about AI crawlers.

If you build with Next.js, also look at streamed content: text inside a Suspense boundary can arrive late in the response inside a hidden element, which is a related but separate issue covered in Next.js and AI search.

Frequently asked questions

Is text in a CSS-hidden tab ignored by crawlers?

Not by crawlers that read the HTML: the text is in the response and can be extracted. Google says tabs and accordions are an acceptable way to save space on mobile. AI operators don't document whether they weigh hidden text differently, so keep your key facts in visible copy.

Does loading="lazy" on images hurt AI search readiness?

No. Native lazy loading leaves the real src and the alt text in the HTML. The risk comes from script-based lazy loading that keeps the real URL out of src until JavaScript runs.

Is a "Read more" button that expands text a problem?

Only if the extra text is fetched when the button is pressed. If the full text is in the HTML and the button just removes a height limit or a hidden attribute, crawlers can read all of it.

Do user-triggered agents like ChatGPT-User run JavaScript?

OpenAI, Anthropic and Perplexity don't document whether ChatGPT-User, Claude-User or Perplexity-User render JavaScript or click page elements. Treat content behind an interaction as unreadable for these fetchers; browser-based agents that a user directs are a separate case.

Is it enough that Google can see my lazy-loaded content?

For Google Search, yes. For AI crawlers that read raw HTML, no: content that appears only after rendering can pass URL Inspection and still be missing from what they receive.

Check your own pages

Start with the templates that hide the most text: product pages with tabs, FAQ pages with accordions and listing pages with infinite scroll. The free check tests one page, with no account, and reports whether its main text is in the raw HTML. If the pattern sits in a shared template, a full audit runs the same checks across up to 500 pages, so you can see every page that uses it.