HeadlessChrome user agent in logs: what it is and who sends it
What the HeadlessChrome user agent means in your logs, where Chrome adds it, which tools send it, why it is easy to remove, and how bot products classify it.
By Crawlito · 9 min read
In short
- HeadlessChrome is the product token Chrome puts in its user agent when it runs without a visible window: Chromium's source prefixes Headless to the Chrome product name when the --headless switch is set.
- In our test, Chrome 157 in --headless mode sent HeadlessChrome/157.0.0.0, and chrome-headless-shell 144 sent HeadlessChrome/144.0.7559.96 plus a HeadlessChrome brand in its client hints.
- Puppeteer, Playwright and Chrome's --user-agent flag can replace the string, so HeadlessChrome only shows up when the operator did not change it.
- No AI search crawler documented by OpenAI, Anthropic, Perplexity or Google uses HeadlessChrome as its name; the token identifies a browser engine, not an operator.
- AWS WAF Bot Control has a SignalAutomatedBrowser rule that blocks requests not from verified bots when they show indicators of an automated browser, which can include your own tests and monitors.
On this page
- What does HeadlessChrome mean in a user agent?
- The two headless modes and what each one sends
- Why the string is easy to change
- Who sends HeadlessChrome to your site
- How to find HeadlessChrome requests in your logs
- How bot management products classify automated browsers
- Frequently asked questions
- Check what a crawler receives
HeadlessChrome in a user agent means the request came from Google Chrome or Chromium running in headless mode, with no visible window, usually under the control of a script. It does not name who is running it. The same token appears for a developer's test suite, an uptime monitor, a screenshot service, a scraper and an AI agent framework built on Puppeteer or Playwright. Because any of those tools can change the string with one line of code, seeing it tells you that someone did not bother to hide it, and its absence proves nothing.
This post covers what the string is and who sends it. For the decision about whether to allow, challenge or block an unknown client, see how to identify an unknown bot user agent.
What does HeadlessChrome mean in a user agent?
A normal desktop Chrome user agent ends with Chrome/<version> Safari/537.36. In headless mode, Chrome swaps the Chrome product token for HeadlessChrome and leaves the rest alone:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/157.0.0.0 Safari/537.36This comes from Chromium's source, in two places. For regular Chrome, user_agent_utils.cc inserts Headless in front of the product name when the --headless switch is present. For the separate headless shell, headless_browser_impl.cc sets the constant kHeadlessProductName to "HeadlessChrome", with a comment saying it is the product name for building the default user agent string. Chrome's own headless Chrome article shows the same token in its example of printing the browser's user agent: "something like" HeadlessChrome/60.0.3082.0.
The token is not an operator or a crawler name. It sits where a browser name normally sits, so it describes the software, not the person or company running it.
The two headless modes and what each one sends
Chrome has had two headless implementations. Chrome's headless mode documentation says the old headless mode was "a separate, alternate browser implementation", that Chrome now has unified headless and headed modes, and that since Chrome 132.0.6793.0 the old mode is only available as a standalone binary called chrome-headless-shell. The older headless Chrome article adds that the new mode became available in Chrome 112 and that chrome-headless-shell binaries exist from Chrome 120.
We checked what each one reports on macOS, by loading a local page that prints navigator.userAgent and navigator.userAgentData.brands:
chrome --headless --dump-dom file:///tmp/ua.html
chrome-headless-shell --dump-dom file:///tmp/ua.html| Binary | User agent product token | Client hint brands |
|---|---|---|
Google Chrome 157, --headless | HeadlessChrome/157.0.0.0 | Not-A.Brand, Google Chrome, Chromium |
| chrome-headless-shell 144 | HeadlessChrome/144.0.7559.96 | Not(A:Brand, Chromium, HeadlessChrome |
Two details are useful when you read logs:
- The version format differs. Regular Chrome reports a reduced version ending in
.0.0.0. The shell reported the full build number. A full four-part build afterHeadlessChrome/points to the shell; a reduced one points to the regular Chrome binary. - The shell can also say it in client hints. The shell listed
HeadlessChromeas a brand innavigator.userAgentData, and Chromium's user agent metadata test expects that brand by default. The same test checks that theSec-CH-UArequest header is sent. The brand list is separate from the user agent string: when we changed the string with--user-agent, the brands stayed as they were. In our test, regular Chrome in--headlessmode did not list HeadlessChrome as a brand.
These are results from one machine with the versions shown. Treat them as what to expect, not as a guarantee for every build.
Which libraries use which mode
- Puppeteer launches headless by default. Its headless modes guide says that before v22 Puppeteer used the old headless mode by default, that the old mode is now
chrome-headless-shell, and that you select it withheadless: 'shell'. - Playwright ships a separate headless shell and, per its browsers page, uses it by default when running headless. You opt into Chrome's new headless mode with the
chromiumchannel.
So an unchanged Playwright script running headless is likely to send the shell's string, with its full build number.
Why the string is easy to change
Every common tool lets the operator set any user agent:
// Puppeteer: the options form is current; the plain string form is marked obsolete
await page.setUserAgent({ userAgent: 'ExampleMonitor/1.0' });
// Playwright
const context = await browser.newContext({ userAgent: 'ExampleMonitor/1.0' });Puppeteer documents this as page.setUserAgent(), which also accepts optional userAgentMetadata for the client hints. Playwright's emulation guide shows the userAgent option and notes that device presets such as "Desktop Chrome" bring their own user agent string. Chrome itself accepts a command-line switch; in our test, --user-agent="ExampleMonitor/1.0" replaced the whole string seen by the page.
That has two consequences:
- HeadlessChrome is an opt-out signal. You see it when the operator left the default in place. An operator who wants to blend in can replace it with a normal Chrome string.
- You cannot use it to block automation. A rule that blocks
HeadlessChromestops honest test scripts and tools that did not change the default, and nothing else. A user agent string is a claim, not an identity.
Who sends HeadlessChrome to your site
Anything that drives Chrome without a window and does not override the user agent. The usual groups, and where to look to tell them apart:
| Group | Typical reason to run headless Chrome | How to tell it apart |
|---|---|---|
| Your own team | End-to-end tests, preview builds, PDF or image generation | Addresses of your CI runners, servers or office |
| Monitoring and testing services | Synthetic checks, screenshots, performance runs | The vendor's published IP list or its own user agent name |
| SEO and site-audit tools | Rendering JavaScript pages before reading them | The tool's documented user agent and addresses |
| Scrapers and price trackers | Reading content that needs JavaScript | Cloud or proxy addresses and no documentation to match |
| AI agent frameworks and hosted browsers | Letting a model open and read pages | Operator headers or signatures, if the platform adds them |
None of these groups is guaranteed to send the default string. Whether a given tool keeps it depends on how its operator configured it, and most vendors do not document it.
Some named examples, from the operators' own documentation:
- Lighthouse changes the user agent by default as part of its emulation. Its emulation documentation describes an
emulatedUserAgentsetting that is on by default, so a Lighthouse run does not necessarily look like plain headless Chrome. - Cloudflare Browser Rendering sends the underlying Chrome version's default user agent when you drive it with Puppeteer, Playwright or CDP, according to its automatic request headers page. The same page says the user agent is not a reliable way to identify it and that it adds a
cf-brapi-devtoolsheader and Web Bot Auth signature headers that cannot be removed. Our post on Cloudflare Browser Rendering's crawler covers its crawl endpoint.
What HeadlessChrome is not
It is not one of the documented AI crawlers. OpenAI, Anthropic, Perplexity and Google publish named user agents for their crawlers and fetchers, such as GPTBot, ClaudeBot, PerplexityBot and Googlebot, and none of them uses HeadlessChrome as its identifier. Google's JavaScript SEO basics say a headless, evergreen Chromium renders pages for Search, but the user agent strings it publishes carry a Chrome/W.X.Y.Z version plus the Googlebot token, not HeadlessChrome. If a request says HeadlessChrome, it did not come from one of those crawlers identifying itself.
It is also not proof of an AI agent. Some AI browser agents look like ordinary Chrome instead, which AI browser agents in server logs covers in detail. Plain HTTP clients, which never run a browser, are a different case again; see python-requests, Scrapy and other library user agents.
How to find HeadlessChrome requests in your logs
Start with a count and the paths it hit. For an nginx or Apache combined log:
# How many requests, and from which addresses
grep -c 'HeadlessChrome' /var/log/nginx/access.log
grep 'HeadlessChrome' /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head
# Which version strings: a full build number suggests chrome-headless-shell
grep -o 'HeadlessChrome/[0-9.]*' /var/log/nginx/access.log | sort | uniq -c | sort -rnThen answer three questions before you decide anything:
- Is it you? Compare the addresses with your CI runners, office network, staging hosts and the monitoring services you pay for. Rule out your own tools before you look further.
- What did it fetch? A burst of HTML pages, scripts and images in the order a browser loads them means a real browser session. Steady requests to one URL every few minutes look like a monitor.
- Does it carry anything verifiable? Look for operator headers such as
cf-brapi-devtoolsor aSignature-Agentheader. A header is still only a claim until a signature or a published IP range confirms it.
If you want the client hint evidence, log Sec-CH-UA next to the user agent. In nginx the variable is $http_sec_ch_ua, because nginx's $http_name variables lowercase the header name and turn dashes into underscores. MDN's Sec-CH-UA reference notes two limits: the header is only available in secure contexts (HTTPS), and the brand list deliberately includes a fake brand such as Not A;Brand, so match the brands you recognize and ignore the rest.
log_format ua_hints '$remote_addr [$time_local] "$request" $status '
'"$http_user_agent" "$http_sec_ch_ua"';
access_log /var/log/nginx/access_hints.log ua_hints;How bot management products classify automated browsers
Bot products generally do not publish how they detect automated browsers. AWS documents more than most for its Bot Control rule group, though it describes "indicators" rather than naming the user agent as the signal:
| Rule | What AWS says it does | Default action |
|---|---|---|
SignalAutomatedBrowser | Inspects requests not from verified bots for indicators that the browser might be automated | Block |
TGT_SignalAutomatedBrowser | Inspects the AWS WAF token of requests not from verified bots for signs of automation; only applies when the request has a token | CAPTCHA |
TGT_SignalBrowserInconsistency | Checks browser interrogation data for inconsistencies; also token-based | CAPTCHA |
SignalNonBrowserUserAgent | Matches user agents that do not seem to come from a web browser | Block |
AWS notes that automated browsers can be used for testing or scraping, and gives monitoring or verifying your own site as an example. The rules without the TGT_ prefix belong to the common protection level, which AWS says detects self-identifying bots including automated browsers and blocks the ones it cannot verify. So a common-level Bot Control setup can block your own headless test runs and monitors unless you make an exception for them. The AWS WAF Bot Control guide shows the pattern for one rule: override it to Count, then add label-based rules after the rule group. Other vendors use their own signals and mostly do not publish them, so check your provider's documentation rather than assuming the same rules.
The TGT_ rules only apply when the request carries an AWS WAF token, which comes from AWS's challenge, CAPTCHA or application SDK running in the client. That is a reminder that some automation detection depends on code running in the browser, not on the log line. That is also why a headless client that runs your scripts is seen differently from a crawler that never runs them; do AI crawlers run JavaScript? explains the split.
Frequently asked questions
Is HeadlessChrome a bot?
It is a browser running without a window, usually under a script's control. Whether that counts as a bot depends on who runs it: your own tests and monitors send it too.
Is HeadlessChrome an AI crawler?
No documented AI crawler from OpenAI, Anthropic, Perplexity or Google uses it as its name. An AI agent framework built on Puppeteer or Playwright could send it if its operator left the default user agent in place.
Should I block HeadlessChrome in robots.txt or my firewall?
A block only stops clients that did not change the default string, which are often your own tools. Decide on verified identity and behavior instead, using the steps linked at the top of this post.
Why does HeadlessChrome show a different version format from normal Chrome?
In our test, regular Chrome in headless mode reported a reduced version such as 157.0.0.0, while chrome-headless-shell reported its full build number. A full build number after HeadlessChrome/ usually means the shell, which Playwright uses by default for headless runs.
Check what a crawler receives
Knowing who sent a request is half the picture. The other half is what your pages return to automated clients, including any challenge or block page from your edge. The free single-page check needs no account and shows what one page returns to the named AI crawlers, including the robots.txt rule each one gets.
Filed under AI crawlers. Topics: server-logs, ai-agents, bot-protection, javascript