Cloudflare 520, 521, 522 and 524 errors: what crawlers see
Cloudflare error 522, 520, 521 or 524 in a crawl report means the edge could not get a page from your origin. See each documented cause and what crawlers do.
By Crawlito · 9 min read
In short
- Cloudflare's 520 to 526 errors are generated at the edge when it cannot get a usable response from your origin, so every visitor and crawler on that request gets a 5xx and no page content.
- Cloudflare documents a 522 when the origin does not answer a TCP SYN within 19 seconds, or does not acknowledge the request within 90 seconds after connecting, and names blocked or rate-limited Cloudflare IPs as the most common cause.
- A 524 means Cloudflare connected but the origin sent no HTTP response within the Proxy Read Timeout, which is 125 seconds by default at the time of writing and can be raised only on Enterprise.
- Google says it ignores content from URLs that return 5xx, slows crawling, and eventually drops URLs that keep failing.
- A bot rule block looks different: errors 1010 and 1020 arrive as 403, rate limiting error 1015 as 429, and challenge pages carry a cf-mitigated header set to challenge.
On this page
Cloudflare errors 520, 521, 522, 523, 524, 525 and 526 mean Cloudflare's edge tried to fetch the page from your origin server and failed. Error 522 is a connection timeout, 521 is a refused connection, 524 is an origin that connected but never answered, and 520 is an empty or malformed reply. Crawlers get the same 5xx status as people do, so they receive no content, and Google treats repeated 5xx responses as a reason to crawl less and, eventually, to drop the URLs. The fix is almost always at the origin or between Cloudflare and the origin, not in your robots.txt or bot settings.
This post covers the origin side. If your report shows 403s, error 1020 or 1010, or a challenge page instead, see Cloudflare error 1010 and 1020. If the crawler cannot even resolve your hostname or complete TLS with Cloudflare, that is a different failure, covered in DNS and TLS errors for AI crawlers.
What each Cloudflare 52x error means
Cloudflare lists each code on its 5xx error reference, with a separate page per error. The causes below are the ones those pages name.
| Error | Cloudflare's title | What happened between Cloudflare and the origin | Causes Cloudflare lists |
|---|---|---|---|
| 520 | Web server returns an unknown error | The origin replied with something Cloudflare could not use | Origin crash, firewall blocking Cloudflare IPs, headers over 128 KB, empty or malformed response, bad HTTP/2 setup |
| 521 | Web server is down | The origin refused the connection | Web server offline, Cloudflare requests blocked or rate limited, nothing listening on port 80 or 443 |
| 522 | Connection timed out | The connection never completed, or the request was never acknowledged | Cloudflare IPs blocked or rate limited, overloaded origin, wrong origin IP in DNS, keepalives off, dropped packets |
| 523 | Origin is unreachable | No network route to the origin's IP | Wrong A or AAAA record, routing problem, overly broad cloud route table |
| 524 | A timeout occurred | Connected, but no HTTP response in time | Long-running query or overloaded server |
| 525 | SSL handshake failed | TLS between Cloudflare and the origin failed | No valid certificate, port 443 closed, no SNI support, no shared cipher suites |
| 526 | Invalid SSL certificate | The origin certificate failed validation in Full (strict) mode | Expired, self-signed or wrong-name certificate, incomplete chain |
525 and 526 concern the hop from Cloudflare to your origin. The certificate a crawler sees on your hostname is Cloudflare's edge certificate, which is why those two errors can appear while a browser shows a valid padlock.
The timeouts Cloudflare documents
Two numbers are worth knowing, both from Cloudflare's pages at the time of writing:
- 522: Cloudflare's 522 page says the error occurs when Cloudflare does not receive a SYN+ACK within 19 seconds of sending a SYN, or, once connected, does not receive an acknowledgment of its request within 90 seconds.
- 524: The 524 page gives a default Proxy Read Timeout of 125 seconds, and a Proxy Write Timeout of 30 seconds that cannot be adjusted. Enterprise customers can raise the read timeout up to 6,000 seconds.
A crawler may give up long before either limit. At the time of writing, Google, OpenAI, Anthropic and Perplexity do not publish crawler timeouts in seconds, so a page that takes 60 seconds may never produce a 524 in your Cloudflare analytics and still fail for the crawler. Slow TTFB and crawler timeouts covers that side.
Why your origin firewall can cause 521 and 522
The most useful line in Cloudflare's 522 page is that confirming your host allows all Cloudflare IP ranges addresses the most common cause. The 520 and 521 pages say the same thing in different words: security software or firewalls at the origin may block or rate limit Cloudflare's IPs.
This matters for crawlers in a way that is easy to miss. When a site is proxied, your origin sees Cloudflare's addresses, not the visitor's. Cloudflare's guide to restoring visitor IPs says the origin logs a Cloudflare IP by default and the real address travels in the CF-Connecting-IP header. A host firewall, a rate limiter or a ban tool that counts requests per source IP is therefore counting Cloudflare's edge addresses. A burst of crawler traffic can push one edge address over a limit, and then everything that edge address carries is refused. That is our inference from how proxying works, not a scenario Cloudflare describes, but it explains why crawler-heavy hours and 52x spikes often line up.
Check three places:
- The host firewall (
iptables,nftables,ufw, a cloud security group). If you restrict the origin to Cloudflare, compare your list against Cloudflare's published ranges; a missing range means some edge locations cannot connect. - Ban and rate-limit tools at the origin such as fail2ban, ModSecurity rules or a security plugin. Make them read the visitor IP from
CF-Connecting-IP, and only trust that header from Cloudflare's ranges. - The hosting provider's own protection. Shared hosts sometimes rate limit at their network edge. Cloudflare's general advice is to contact the host first, with the error code, the time and timezone, and the URL.
For nginx, Cloudflare documents the realip module approach:
# Repeat set_real_ip_from for every range in https://www.cloudflare.com/ips-v4 and /ips-v6
set_real_ip_from 173.245.48.0/20;
real_ip_header CF-Connecting-IP;With that in place, per-IP limits and logs apply to the real client rather than to Cloudflare. If you then want to limit a specific crawler, do it with a Cloudflare rule rather than an origin ban, so the response is a clear 429 instead of a dropped connection.
What Google does with Cloudflare 5xx responses
Google's page on how HTTP status codes affect its crawlers does not list 520 to 526 by number. It does make category-level statements about 5xx:
- 5xx and 429 errors make Google's crawlers temporarily slow down.
- Any content Google receives from a URL that returns 5xx is ignored.
- Already indexed URLs are kept at first but eventually dropped if the errors persist.
- Once the server returns 2xx again, Google gradually increases the crawl rate.
Its page on DNS and network errors adds that timeouts and connection resets are handled like 5xx errors, crawling slows immediately, and unreachable indexed URLs can be removed within days. With Cloudflare in front, an origin timeout reaches Googlebot as a 522 or 524 status rather than as a network error, so the 5xx rules are the ones to plan around.
The Search Console Crawl Stats report is where this shows up. Its crawl responses table has a Server error (5XX) row whose chart shows approximately when the errors occurred, and the host status section flags server connectivity problems. A failed robots.txt fetch with a 5xx is more serious: the same help page says Google will slow or stop crawling the site until it gets an acceptable robots.txt response, and that crawling stops for the first 12 hours of failures. If /robots.txt is served by the same failing origin, a 522 storm affects the whole site. For how 5xx appears in the page indexing report, see server error (5xx) in Search Console.
OpenAI, Anthropic and Perplexity do not document how their crawlers treat 5xx responses at the time of writing. What is certain is that a 52x response contains no page content for them to read.
How to tell an origin error from a bot rule block
The first question is whether Cloudflare refused the request on purpose or failed to reach the origin. The status code and headers usually answer it.
| What you see | Likely source | Where to look |
|---|---|---|
| 520 to 526 | Cloudflare could not get a usable response from the origin | Origin logs, host firewall, Origin Analytics |
| 403 with a 1xxx code in the body | A Cloudflare security feature blocked the request | Security Events |
| 429, or error 1015 | A rate limiting rule | Security Events, rate limiting rules |
Any status with cf-mitigated: challenge | A challenge page was served | Security Events, bot and WAF settings |
| 500, 502, 503 or 504 with your own error page | Your origin or app answered with an error | Application logs |
Cloudflare's challenge detection page says the cf-mitigated header is set to challenge on every challenge page type, and that challenge is its only valid value. A 52x response is not a challenge, so it will not carry that header.
Then confirm with two requests, one through Cloudflare and one straight to the origin:
# Through Cloudflare, with a crawler user agent
curl -s -o body.html -D - -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
https://www.example.com/ | grep -iE "^HTTP|^cf-ray|^cf-mitigated"
grep -ioE "error( code)? ?5[0-9]{2}" body.html | head -3
# Straight to the origin, bypassing Cloudflare (203.0.113.10 is your origin IP)
curl -s -o /dev/null -w "%{http_code} %{time_starttransfer}s\n" \
--resolve www.example.com:443:203.0.113.10 https://www.example.com/Read the results this way:
- The edge returns 522 and the direct request also hangs: the origin is down or overloaded.
- The edge returns 522 but the direct request works: something between Cloudflare and the origin is dropping Cloudflare's traffic. That is usually the firewall or rate limiter described above. If your firewall only admits Cloudflare, the direct test will fail from your laptop; run it on the server itself against
127.0.0.1instead. - The edge returns 520 but the direct request returns 200: Cloudflare's 520 page says an origin 200 served as a 520 points to a malformed response, such as oversized headers or an early connection close. Check header size with
curl -sIand look for cookie bloat.
Record the cf-ray value. Cloudflare asks for it when you escalate, and you can filter logs by Ray ID.
How to fix each error and check the fix
| Error | First fix to try | How to confirm |
|---|---|---|
| 520 | Read origin error logs for crashes; trim response headers; if the origin's HTTP/2 is broken, turn off HTTP/2 to Origin | Direct and proxied requests both return 200 |
| 521 | Start the web server; allow Cloudflare ranges; make sure it listens on 443 for Full or Full (strict) | ss -ltn shows the port; proxied request returns 200 |
| 522 | Allow all Cloudflare ranges in the firewall and rate limiters; check the origin IP in Cloudflare DNS; enable keepalives | 522 count falls in Cloudflare analytics |
| 523 | Correct the A or AAAA record; fix routes that capture Cloudflare's ranges | Traceroute from origin to a Cloudflare IP completes |
| 524 | Move slow work out of the request; poll long jobs; serve exports from a DNS-only subdomain | Origin response times well under the read timeout |
| 525 | Install a valid certificate or a Cloudflare Origin CA certificate; open 443; support SNI | Proxied HTTPS request returns 200 |
| 526 | Fix expiry, hostname or chain; or add the certificate to the Custom Origin Trust Store | Full (strict) works without errors |
Cloudflare's error analytics live under HTTP Traffic, where you can filter by edge or origin status code. Cloudflare notes they are based on a 1% sample, so use them for trends, not exact counts.
Two Cloudflare features soften the impact but do not fix the cause. Always Online, available on all plans, only activates when Cloudflare cannot connect to the origin, not when the origin answers with its own 5xx. It then serves a stale copy from Cloudflare's cache or static portions of the page from the Internet Archive, if either exists. Planned downtime is a different case: return a proper 503 with Retry-After, as described in HTTP 503 for maintenance, rather than letting Cloudflare report a 521.
Frequently asked questions
Does a Cloudflare 522 mean Cloudflare is down?
Usually not. Cloudflare's documentation describes 522 as Cloudflare failing to complete a connection to your origin, and names blocked or rate-limited Cloudflare IPs as the most common cause.
Will a 52x error remove my pages from Google?
A short burst should not. Google says it keeps already indexed URLs at first, slows crawling, and drops URLs that persistently return server errors.
Can I fix 52x errors by allowing Googlebot in Cloudflare?
No. Bot and WAF rules decide whether Cloudflare forwards a request; a 52x happens after that, when Cloudflare cannot get a response from the origin.
Why do 522 errors appear mostly when crawlers visit?
Crawler bursts can overload a small origin, and origin tools that rate limit per IP see Cloudflare's addresses rather than the crawler's. Make those tools read CF-Connecting-IP and allow Cloudflare's ranges.
Is a 524 the same as a slow page?
A 524 means the origin sent no response within Cloudflare's read timeout, 125 seconds by default at the time of writing. Crawlers may abandon a page much sooner, so a page can be too slow for them without ever producing a 524.
Check what crawlers get from your pages
52x errors come and go with load, so a single test can miss them. Run the free check on your key pages after a fix to confirm AI crawlers get a 200 and real content. If errors keep returning, an audit reviews the edge settings, origin responses and robots.txt together, with evidence for each finding.
Filed under CDN and bot protection. Topics: cloudflare, http-status, cdn, troubleshooting, google