Skip to content

Googlebot file size limit: 2 MB of HTML and what gets cut off

Googlebot reads the first 2 MB of an HTML file, uncompressed, and ignores the rest. See what gets truncated, why inline data causes it and how to measure your pages.

By Crawlito · 6 min read

In short

  • Googlebot for Google Search crawls the first 2 MB of a supported file type such as HTML, and Google says it stops the fetch and only sends the downloaded part for indexing consideration.
  • The limit is applied to uncompressed data, so gzip or Brotli on the wire does not raise it.
  • PDFs have a separate 64 MB limit, and CSS and JavaScript files referenced by a page are fetched separately, each with its own per-URL limit.
  • Google's general crawler documentation gives 15 MB as the default for its crawlers and fetchers, with smaller limits such as Googlebot's 2 MB set per crawler.
  • Inline base64 images, large inline JSON state and inline CSS or JavaScript are the usual reasons a page passes 2 MB, and they can push text and structured data past the cutoff.
On this page

Googlebot reads the first 2 MB of an HTML page and ignores everything after that point. Google's Googlebot documentation says the limit is applied to the uncompressed file, that PDFs get 64 MB, and that once the cutoff is reached Googlebot stops the fetch and sends only the part it downloaded for indexing. Most pages are far below 2 MB. The ones that are not usually carry inline data: base64 images, serialized app state or very large inline scripts.

If your page is over the limit, nothing visible goes wrong. The page still loads, Search Console may still show it as indexed, and the missing part is simply never considered. That makes it worth measuring rather than guessing.

What are Googlebot's current file size limits?

Google documents the limits in two places, and the numbers differ because they describe different things.

WhatLimitSource
Googlebot, HTML and other supported text filesFirst 2 MB, uncompressedGooglebot
Googlebot, PDF filesFirst 64 MBGooglebot
CSS, JavaScript and other referenced resourcesFetched separately, each bound by the same per-file limitGooglebot
Google's crawlers and fetchers in generalFirst 15 MB by default; individual crawlers may set smaller limitsOverview of Google crawlers

The 15 MB figure is the one many older articles quote for Googlebot. In early 2026 Google reorganised its crawler documentation so that 15 MB is described as the default for Google's crawlers, while Googlebot, the crawler for Google Search, has the smaller 2 MB limit. Google's March 2026 Search Central post, Inside Googlebot, explains the per-URL counting and says the limit is not set in stone and may change over time. Check the documentation again before quoting the figure in a report.

What gets truncated when a page is over 2 MB?

Everything after the cutoff byte. Google does not reject the page; it uses the first part. What you lose depends on what sits at the end of your HTML:

  • Main text further down the page. Long articles, product descriptions below a large gallery, or reviews at the bottom.
  • Links. Footer links and pagination are at the end of most templates. If they fall past the cutoff, Googlebot cannot use them to discover other pages from that URL.
  • Structured data. Many frameworks print JSON-LD at the end of <body>. If it is past the cutoff, it is not read.
  • Content in a late hydration payload. Some server-rendered apps put the page's data in a script near the end of the HTML. If the visible HTML is complete, losing the payload may not matter for indexing; if the text only exists in that payload, it does.

The March 2026 post gives the practical rule: keep critical elements such as the title, meta tags, canonical link and essential structured data near the top of the document, so they are unlikely to fall below the cutoff.

What makes HTML bigger than 2 MB?

HTML text alone rarely reaches 2 MB. A 5,000-word article is roughly 30 to 40 KB of text. Pages pass the limit because of things embedded in the HTML that are not text:

CauseWhy it is largeFix
Inline base64 imagesEach image is encoded inside the HTML, about a third larger than the binaryServe images as files with <img src>
Inline SVG repeated per itemIcons copied into every card in a long gridUse an external sprite or <img>
Serialized app stateFrameworks embed the full data set for hydration, sometimes including fields the page never showsTrim the payload to what the page needs
Inline CSS and JavaScriptBuild settings that inline whole bundlesMove to external files, which get their own limit
Huge navigation menusMega menus with thousands of links printed on every pageReduce the menu or load secondary levels on demand
Unpaginated listsEvery product or comment rendered into one pagePaginate with crawlable links

Google's post names the first and fourth of these directly: bloated inline base64 images and massive blocks of inline CSS or JavaScript can push your text or structured data past the cutoff. Because referenced CSS and JavaScript files are fetched separately with their own counter, moving them out of the HTML removes their weight from the page.

How to measure your page's HTML size

Measure the uncompressed HTML the server sends, not the transfer size in your browser's network panel, and not the rendered DOM.

Check the total size

bash
# Decompressed size of the HTML, in bytes
curl -s --compressed https://www.example.com/products/widget | wc -c

--compressed asks for gzip or Brotli and decodes it, so wc -c counts uncompressed bytes, which is what Google's limit applies to. Anything under about 1.5 MB leaves room; anything near or over 2,000,000 bytes needs work. Treat 2 MB as roughly two million bytes; Google's documentation does not say whether it means decimal or binary megabytes, so leave a margin.

Find where the important content sits

The total is only half the question. Find the byte offset of the things you care about:

bash
curl -s --compressed https://www.example.com/products/widget -o page.html
grep -b -o 'application/ld+json' page.html | head
grep -b -o '<footer' page.html
grep -b -o 'data:image/[a-z]*;base64' page.html | wc -l

grep -b prints the byte offset before each match. If your JSON-LD starts at byte 2,300,000, Googlebot does not see it. The last command counts inline base64 images.

Find the biggest inline blocks

A quick script shows which inline scripts and styles take the most space:

python
import re, sys
html = open(sys.argv[1], encoding="utf-8", errors="replace").read()
blocks = re.findall(r"<(script|style)[^>]*>(.*?)</\1>", html, flags=re.S | re.I)
for tag, body in sorted(blocks, key=lambda b: -len(b[1]))[:5]:
    print(tag, len(body.encode("utf-8")), body[:60].replace("\n", " "))
print("total bytes:", len(html.encode("utf-8")))

Run it with python3 inline-size.py page.html. If one script holds most of the page, you have found the problem.

Search Console's URL Inspection tool shows the HTML Google has for a URL, which is a useful cross-check when a block near the end seems to be missing. Our URL Inspection guide explains what each view shows.

How to fix a page that is over the limit

Fix the cause, not the symptom. In order of usual payoff:

  1. Move images out of the HTML. Replace base64 src values with image files. This alone often fixes the problem.
  2. Trim serialized state. In Next.js, Nuxt and similar frameworks, pass only the fields a page renders. A product listing does not need every variant's full record in the HTML.
  3. Externalise CSS and JavaScript. Turn off build options that inline entire bundles.
  4. Move critical tags up. Put the title, meta description, canonical and JSON-LD in the <head> so they are within the first few kilobytes.
  5. Paginate long lists. Use real links to further pages instead of rendering hundreds of items at once.

After the change, measure again with the same commands and record the new size and the byte offset of your JSON-LD and main heading.

Do AI crawlers have file size limits?

Some do, and most do not say. Meta's crawler documentation says Open Graph properties must be within the first 1 MB of a page. OpenAI, Anthropic and Perplexity do not state a page size limit in their crawler documentation at the time of writing. Unknown is not the same as unlimited, so the same advice applies: keep pages lean and put important content early.

Google's AI Overviews and AI Mode are part of Google Search, and Google says Googlebot is the crawler that controls access for them, so the 2 MB limit is the one to plan for there. Raw versus rendered HTML covers the related problem of text that only exists after JavaScript runs, and client-side rendering covers pages that send almost no HTML at all.

Frequently asked questions

Is the Googlebot limit 2 MB or 15 MB?

For Google Search, 2 MB. Google's crawler overview gives 15 MB as the default for its crawlers in general and says individual crawlers may set smaller limits, with Googlebot's 2 MB as the example.

Does gzip or Brotli compression help with the limit?

No. Google says the limit is applied to the uncompressed data, so compression reduces transfer time but not the size Google counts.

Do my CSS and JavaScript files count towards the page's 2 MB?

No, as long as they are external files. Google says referenced resources are fetched separately, each with its own per-URL limit. Inline CSS and JavaScript are part of the HTML and do count.

What is the limit for PDFs?

Googlebot crawls the first 64 MB of a PDF file. See PDF documents and AI search for other PDF issues.

Check your page size and structure

The free check fetches a page's raw HTML and reports whether its main content, headings and structured data are present in what the server sends. For a template-wide view, including page weight across many URLs, an audit samples the site and lists the fixes in order.

Filed under Rendering and content. Topics: google, googlebot, html, rendering, troubleshooting