Client-side rendering, SEO and AI: the empty HTML problem

A client-side rendered (CSR) site sends every visitor the same nearly empty HTML file: a <div id="root"></div> and a script tag. A browser runs the script and builds the page. A crawler that does not run JavaScript keeps the empty file, so it sees no headings, no body text and often no links. In the most recent public measurement, the major AI crawlers fell in that second group, which makes CSR a common reason a site that works fine for people reads as blank to AI search tools.

This guide shows what that empty HTML looks like, how to recognise the symptoms, the fixes ranked by effort, and how to confirm the fix worked.

What does a client-side rendered page look like to a crawler?

Here is the raw HTML a typical single-page app built with Vite and React returns for every URL. It is what you see with view-source, not in the browser's element inspector.

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Acme</title>
    <script type="module" crossorigin src="/assets/index-4f2a9c.js"></script>
    <link rel="stylesheet" crossorigin href="/assets/index-9b1d3e.css">
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>

Everything a visitor reads (the product name, the pricing table, the FAQ, the navigation) lives inside that JavaScript bundle. It reaches the page only when a browser downloads the bundle, runs it, often calls an API, and writes the result into #root.

A crawler that parses HTML without executing scripts gets one word of text from that file: the title, "Acme". The same is true for every route, because a CSR app serves the same file at /, /pricing and /blog/some-post and lets the client-side router decide what to draw. Raw HTML versus rendered HTML is the core distinction here; Raw HTML vs rendered HTML covers it in more depth.

Why do non-rendering crawlers see nothing?

Rendering a page means running a full browser engine: downloading scripts, executing them, waiting for network calls and building the DOM. That costs far more than fetching HTML, and the major AI crawlers measured so far skip it.

The main public data comes from Vercel and MERJ's The rise of the AI crawler, published in December 2024. It found that none of the major AI crawlers rendered JavaScript at the time, naming OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot, Anthropic's ClaudeBot, Meta-ExternalAgent, Bytespider and PerplexityBot. ChatGPT's and Claude's crawlers did download JavaScript files (11.50% and 23.84% of their requests) but did not execute them. Treat this as a snapshot from that date, not a permanent rule, and see Do AI crawlers run JavaScript? for the full picture by crawler.

Google is the main exception. Google Search runs JavaScript with an evergreen version of Chromium, queuing pages for rendering after the first crawl. Vercel's analysis also reported that Gemini uses Googlebot's infrastructure and that Applebot renders through a browser-based crawler, while Apple's own page says only that Applebot "may render" pages. Even so, Google says server-side rendering or pre-rendering "is still a great idea", because it makes a site faster for users and crawlers and not all bots can run JavaScript.

Platforms that serve rendered HTML say the same thing. Wix, for example, tells its users that most AI crawlers do not run JavaScript and read only the HTML the server sends.

Symptoms of the empty HTML problem

You rarely get an error message. You get a pattern of small signs:

• View-source shows an empty root. Press Ctrl+U (Cmd+Option+U on a Mac) and search for a sentence from the page. If it is not there, it is not in the raw HTML.
• Google indexes the page but AI tools don't seem to know it. Search Console's URL Inspection shows a fine rendered page, because Google rendered it. Assistants describe your product from third-party sites, or say they could not read the page.
• Every URL has the same title and description. The client sets document.title after load, so the raw HTML for every route carries the homepage's tags.
• Link previews are generic. Link-preview fetchers from chat and social apps generally read only the raw HTML, so shared links show the site name instead of the page title.
• Logs show bots fetching .js files and few deep pages. Crawlers request the bundle but rarely reach other routes, because there are no <a href> links in the HTML to follow; a sitemap becomes their only map of the site. How to find AI crawler visits in your server logs explains how to filter for them.
• Deep links return odd status codes. Without a server rewrite, /pricing returns a 404 from the host; with a catch-all rewrite, a mistyped URL returns 200 with the same shell.

How to check your page in two minutes

Fetch the page the way a non-rendering crawler would: one HTTP request, no JavaScript. First save this short Python script as wordcount.py. It counts the visible words in an HTML document, ignoring scripts, styles and templates.

import sys
from html.parser import HTMLParser

class Text(HTMLParser):
    skip, words = 0, 0
    def handle_starttag(self, tag, attrs):
        if tag in ("script", "style", "template"): self.skip += 1
    def handle_endtag(self, tag):
        if tag in ("script", "style", "template"): self.skip = max(0, self.skip - 1)
    def handle_data(self, data):
        if not self.skip: self.words += len(data.split())

p = Text(); p.feed(sys.stdin.read()); p.close()
print(p.words, "visible words")

Then fetch the page with GPTBot's documented user-agent string from OpenAI's crawler page and pipe the response into it:

curl -s -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
  https://example.com/pricing | python3 wordcount.py

A CSR shell typically returns a single-digit count. A rendered article returns hundreds. Then search for a specific phrase you expect, such as a price or a heading:

curl -s https://example.com/pricing | grep -c "Starter plan"

To compare against what a browser builds, dump the rendered DOM with headless Chrome (this is the macOS path; on Linux the binary is usually google-chrome) and run the same count on it:

"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
  --headless --dump-dom https://example.com/pricing > rendered.html
python3 wordcount.py < rendered.html

A large gap between the raw and rendered counts is the empty HTML problem in one number. Sending a crawler's user-agent string does not reproduce IP-based firewall rules, so this tests rendering, not access. How to see your page as an AI crawler, with curl has more variations.

The fixes, ranked by effort

1. Server-side rendering or static generation

This is the real fix. The server sends HTML that already contains the page's text, links and metadata, and JavaScript then hydrates it to make it interactive. Google's documentation recommends server-side rendering, static rendering or hydration over workarounds.

For a React app this usually means moving to a framework that renders on the server, such as Next.js or React Router in framework mode. Vue has Nuxt, Svelte has SvelteKit, and Astro outputs static HTML by default. Use static generation for pages that change rarely (marketing pages, docs, blog posts) and server rendering for pages that depend on the request.

SSR does not cover everything automatically. Data fetched inside useEffect or loaded only after hydration is still missing from the first response. In the Next.js App Router, content inside a Suspense boundary that resolves at request time is streamed later in the same response inside a hidden element, and how each AI crawler handles that is not documented. At the time of writing, Next.js sends one fully formed HTML document instead of a stream only to bots on its htmlLimitedBots list, and the default list does not include GPTBot, OAI-SearchBot, ClaudeBot or PerplexityBot. Keep the main text in the initial server render, outside Suspense boundaries that wait on request-time data. SSR, SSG and CSR compared for AI search readiness goes through the trade-offs.

2. Prerendering

Prerendering runs your app in a headless browser ahead of time and saves the resulting HTML for each route. You can do it at build time, with a step that visits a list of routes and writes one HTML file per route, or with a service that renders pages on request and caches them.

Build-time prerendering works well when routes are known and content changes on deploy. It struggles with thousands of product pages or content that changes hourly. A service that only serves rendered HTML to bots is dynamic rendering, which Google now calls a workaround, not a recommended solution. Google does not treat it as cloaking as long as the content is similar to what users get. If you go that way, serve the same prerendered HTML to everyone where you can, and keep the bot list current, because a new AI crawler you have not listed gets the empty shell. Prerendering and dynamic rendering: when they help covers the setup in detail.

3. Put key text in the HTML template

If neither of the above can ship this month, edit index.html so the shell is not empty. Put a real title and meta description in the head, and inside the root element add an H1, a short paragraph that says what the site is, and plain links to your main sections.

<div id="root">
  <h1>Acme: invoicing software for small agencies</h1>
  <p>Acme sends invoices, tracks payments and reminds late clients. Plans start at $12 a month.</p>
  <nav>
    <a href="/pricing">Pricing</a>
    <a href="/features">Features</a>
    <a href="/blog">Blog</a>
  </nav>
</div>

The first time the app calls root.render, React clears the existing HTML inside the root before rendering, so users don't see this block twice, though they may glimpse it while the bundle loads. The limits are serious: one template serves every route, so every URL carries the same text, and /pricing still shows no prices to a crawler. Keep the template text accurate and consistent with the app, since text that differs from what users see causes its own problems. Treat this as a stopgap, not a fix.

How to verify the fix

Run these checks on a sample of real URLs after deploying, not only the homepage:

1. Word count. Rerun the curl and wordcount.py command on each URL. The raw count should be close to the rendered count from headless Chrome.
2. Key facts. grep for a heading, a price and a sentence from the middle of the page. Each should appear in the raw response.
3. Per-route metadata. Check that each URL returns its own <title>, meta description and canonical tag in the raw HTML.
4. Links. Confirm the navigation and in-body links are real <a href="..."> elements, so crawlers can discover other pages.
5. Status codes. Run curl -s -o /dev/null -w "%{http_code}\n" on a real deep link (expect 200) and on a made-up one (expect 404).
6. Logs over the following weeks. Look for AI crawlers requesting HTML pages beyond the homepage, not only script files.

Frequently asked questions

Does client-side rendering hurt Google SEO?

Less than it hurts AI search, because Google renders JavaScript. Rendering is queued and can be delayed, though, and Google still recommends server-side rendering or pre-rendering.

Is a loading spinner in the HTML enough?

No. A spinner or "Loading..." text gives a crawler nothing to read about the page. What matters is whether the page's actual text is in the first HTML response.

Do user-triggered agents like ChatGPT-User render JavaScript?

Vercel and MERJ's 2024 data listed ChatGPT-User among the OpenAI agents that did not render JavaScript. Claude-User and Perplexity-User were not in that list, and OpenAI's and Anthropic's crawler pages do not describe JavaScript rendering, so don't rely on any user-triggered agent to run your scripts.

Will server-side rendering get my site cited in AI answers?

No one can promise that. Server-side rendering removes a technical barrier so AI crawlers can read your pages; whether an assistant cites them depends on many things outside your site.

Can I fix only some pages?

Yes. Start with the pages you most want AI tools to read: the homepage, product or service pages, pricing and key articles. Many frameworks let you render those on the server and leave logged-in app screens client-side.

Next step

Fetch your three most important URLs with the commands above and compare the raw and rendered word counts. For a quick check without the command line, the free Crawlito check reports whether one page's main text is in the raw HTML, with no account needed; a full audit runs the same checks on up to 500 pages and compares the raw and rendered version of each.