Sucuri firewall blocking crawlers: settings and Access Denied
Sucuri firewall blocking crawlers? Read the block ID on the Access Denied page, check the bot filter, DDoS and geo options, then allow crawlers safely.
By Crawlito · 9 min read
In short
- Every branded Sucuri block page carries a block ID, such as BNP003 or GEO02, that names the rule which fired; look it up on Sucuri's public WAF signatures list before changing any setting.
- Sucuri's Emergency DDoS protection blocks visitors whose browsers lack JavaScript, except major search engines, and Sucuri says to switch it off once traffic is back to normal.
- The aggressive bot filter blocks empty user agents, ones starting with PHP/ and improper user agents from common browsers, and is reported as BNP004, BNP005 or BNP006.
- Geo Blocking set to View stops GET requests from a country (GEO02), which can stop crawlers that connect from that country; set to Post it stops only POST requests (GEO01).
- A blank screen instead of a branded page is an IDS block, which Sucuri says lasts 20 minutes and can follow repeated 30x, 40x or 50x responses in a short period.
On this page
- Read the block ID on the Access Denied page
- Security options that affect bots
- Geo Blocking, blacklists and blocked user agents
- How to find the blocked request in Audit Trails
- Blank screens: IDS blocks after many errors
- How to allow crawlers without switching protection off
- How to test from outside after a change
- Frequently asked questions
- Check what crawlers get from your site
If the Sucuri Website Firewall is blocking a crawler, the Access Denied page already tells you why: it shows a block ID, and each ID maps to one rule on Sucuri's public WAF signatures list. The settings most likely to catch a crawler are the bad bot signature, the aggressive bot filter, Emergency DDoS protection, Geo Blocking and IP blacklists. Find the ID first, change only the setting it points to, and keep the firewall on.
Sucuri is run by GoDaddy: the footer of sucuri.net names GoDaddy Mediatemple, Inc., doing business as Sucuri. Facts below come from Sucuri's documentation and signature pages, read on 15 November 2026. Menu names can differ between dashboard versions, so confirm them in your own account.
Read the block ID on the Access Denied page
Sucuri's troubleshooting guide separates two kinds of block. Branded block messages display a block ID you can look up. A blank screen with no message is an IDS block, covered further down.
These are the signatures most likely to catch a crawler, with Sucuri's own descriptions:
| Block ID | Sucuri's title | What Sucuri says it means |
|---|---|---|
BNP003 | Bad bot access denied | Malicious bots blocked by user agent; examples given are MJ12bot and Sogou web spider |
BNP004, BNP005, BNP006 | Request blocked, aggressive bot filter | The aggressive bot filter option fired |
BNP002 | Scanning tool blocked | Use of automated tools to scan the site |
FBP006 | Fake bot access | Access by fake bots was detected and blocked |
INA154 | Missing user agent | A request with no user agent |
DDOS21, ddos22 | DDOS attempt blocked | A DDoS attack was blocked (DDOS21), or a request matched anti-DDoS rules (ddos22) |
GEO02 | GeoIP Block | A country is blocked from making GET requests |
GEO01 | GeoIP Block | A country is blocked from making POST requests |
BLACK02, PBI009 | Blacklisted IP address, Blacklisted IP | The visitor's IP is blacklisted and not authorized to view the site |
IPB17 | IP Address not whitelisted | Admin panel access from an IP that is not allowed |
The signature pages give a title and a one-line description, not the exact matching logic. The BNP003 page, for example, names two example bots but does not list every user agent it treats as bad, and the FBP006 page does not say which bots it checks or how. So read the ID as "which rule", then use the dashboard to see which request triggered it.
If the block came from a crawler rather than from your own browser, you will not see the page yourself. Check the firewall's records instead, as shown in the section on Audit Trails below.
Security options that affect bots
Sucuri's Security Options page lists the advanced settings. Four of them can change what a crawler gets:
| Option | What Sucuri says it does | Effect on crawlers |
|---|---|---|
| Aggressive bot filter | Blocks invalid user agents: empty ones, ones starting with PHP/, and improper user agents from common browsers | Catches badly formed bot user agents; Sucuri does not publish how it judges "improper" |
| Enable Emergency DDOS protection | HTTP flood protection that blocks visitors whose browsers lack JavaScript, except major search engines | Blocks any bot that does not run JavaScript and is not treated as a major search engine |
| Block anonymous proxies and the top three attack countries | Blocks visitors from China, Russia and Turkey and users of anonymous proxies from registering, commenting or logging in | Visitors can still view content, so page fetches should still work |
| Advanced evasion detection | Turns on extra evasion signatures | Sucuri says it may need to be off for non-ASCII URLs or Google Analytics and Ads tracking parameters |
Two points need care.
Emergency DDoS protection. Sucuri says to use it only during an attack, because it may block legitimate visitors, and to turn it off once traffic is normal. The documentation does not say which crawlers count as major search engines, and it does not mention AI crawlers. Many crawlers fetch HTML without running JavaScript, so treat any bot outside that unnamed list as blocked while the option is on. If you switched it on during an attack months ago, this is the first setting to check.
The aggressive bot filter. Google, Microsoft, OpenAI, Anthropic and Perplexity publish full user agent strings for their crawlers, so the empty and PHP/ rules should not apply to them. Sucuri does not document how it decides that a browser-like user agent is "improper", so if you see BNP004 to BNP006 against a crawler you want, the filter is the cause and the dashboard is the evidence.
To change either option, Sucuri's IP allowlist page gives the path for this panel: the Security tab, then Advanced Security Options, then Save Advanced Security Options after you change a checkbox.
Geo Blocking, blacklists and blocked user agents
The Access Control area holds the rules you or a previous admin added by hand. Sucuri's country blocking page describes two Geo Blocking modes:
- View: visitors from the chosen countries cannot see the site. This is
GEO02, a block on GET requests, and it stops crawlers too. - Post: visitors can see the site but cannot post, register, buy or comment. This is
GEO01. Crawlers fetch pages with GET, so this mode should not affect normal crawling.
The main AI crawlers connect from cloud data centers, most of them in the US, so a View block on the US, or an allow-only-my-market setup, can shut them out. The geo-blocking guide shows where each crawler connected from when we checked and why these blocks are silent. Sucuri also notes that IP ownership changes every day, so geolocation is accurate but not infallible.
Then read the Block User Agents list. Sucuri's user agent page says a block stops any traffic whose user agent contains the name you entered. That is a substring match, so a short entry such as bot or crawler matches Googlebot, Bingbot, GPTBot, ClaudeBot and every other crawler with that word in its token. Replace broad entries with the exact token you meant to block, for example MJ12bot.
Finally, check the IP blocklist for old ranges. A range added during an attack can overlap a cloud network that a crawler uses.
How to find the blocked request in Audit Trails
Requests the firewall blocks never reach your server, so your origin logs will not show them. Sucuri's Audit Trails page in the dashboard lists blocked requests. Its allow a URL path page links to it at waf.sucuri.net/?audit and describes selecting a blocked URL there.
Look for these in each entry:
- The block ID, to match it to the table above.
- The user agent, to see which crawler it claimed to be.
- The IP address, to check whether it belongs to that crawler's operator. A user agent is only a claim; verifying real AI crawlers shows how to confirm one against its published IP ranges or DNS.
If the IP is not the operator's, the block was correct and should stay. If it is, change the setting the block ID points to.
When you do compare with origin logs, remember that every request reaches your server from a Sucuri IP. Sucuri's same IP for all users page explains how to restore the real visitor address: PHP applications read it from HTTP_X_SUCURI_CLIENTIP, and Apache, nginx and LiteSpeed can take it from X-Forwarded-For once Sucuri's firewall ranges are set as trusted proxies. Without that, every crawler looks like it comes from the firewall.
Blank screens: IDS blocks after many errors
A crawler can be blocked without any branded page. Sucuri's troubleshooting guide says an IDS block shows a blank screen and lasts 20 minutes. It can follow too many prohibited actions from one IP, such as repeated 30x, 40x or 50x responses in a short period, or several failed logins in a few minutes.
That matters for crawlers. A crawler working through old URLs after a redesign, or a redirect chain on every page, produces exactly that pattern. Sucuri says allowlisted IPs are exempt from IDS enforcement unless it sees severe host-level anomalies, such as high 5xx error rates or resource exhaustion. Fixing the broken links and redirect chains removes the trigger, which is better than exempting every crawler.
How to allow crawlers without switching protection off
Change one thing at a time and note what you changed.
- Fix the setting the block ID names. Switch off Emergency DDoS protection if the attack is over, remove a broad entry from Block User Agents, or change a country from View to Post if you only need to stop form spam.
- Allow verified IP ranges, not user agents. In Access Control, enter a range in Allow IP Addresses and select Allow. Sucuri accepts CIDR notation from /16 to /32 and advises against broad ranges. Take the ranges from each operator's published list, as described in the AI crawler IP allowlist guide, and refresh them, because they change.
- Know what an IP allowlist does not cover. Sucuri's public docs, at the time of writing, document allowing IPs and URL paths but not a user agent allowlist. A user agent exemption would also let any scraper that copies the name through. Allowlisted IPs remain subject to IDS enforcement in the cases Sucuri describes above.
- Use Allow URL Paths only as a last resort. Sucuri says it opens that path to all users, and that it does not bypass
BLACK02blocks or JavaScript challenges.
Leave the aggressive bot filter on unless the dashboard shows it hitting a crawler you need. Most of what it blocks has no business on your site.
How to test from outside after a change
First, confirm the firewall is answering. Sucuri's cache headers article shows a server: Sucuri/Cloudproxy header and an x-sucuri-id header on responses that pass through it:
curl -s -o /dev/null -D - https://example.com/ | grep -iE '^(HTTP|server|x-sucuri)'Then request the page with a crawler user agent and look for a block ID in the body. The string below is the OAI-SearchBot example from OpenAI's crawler documentation; OpenAI notes the version number may change.
UA="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot"
curl -s -A "$UA" https://example.com/ | grep -oE '(BNP|FBP|GEO|DDOS|ddos|BLACK|PBI|INA)[0-9]{2,3}' | sort -uNo output and a normal page means your request passed. It runs from your IP, not the crawler's, so it does not prove the crawler gets through: an IP or geo rule treats your request differently, and a request copying a crawler's name from the wrong IP may be what FBP006 is designed to catch. The Audit Trails view stays the evidence.
To tell whether the firewall or your server causes a block, compare with a request that skips the firewall. Sucuri's bypass guide does this with a hosts file entry pointing at your hosting IP; curl --resolve does the same for one request:
curl -s -o /dev/null -w "%{http_code}\n" -A "$UA" \
--resolve example.com:443:203.0.113.10 https://example.com/Replace 203.0.113.10 with the address on the dashboard's Hosting IP Address page. If you enabled Firewall Bypass Prevention, direct connections can return a 403, so add your own IP to the server rule first, as the guide describes. If the response headers do not show Sucuri at all, another layer is in play, and the WAF headers guide helps identify it. For WordPress plugins that block on top of the firewall, see WordPress security plugins and crawlers; for JavaScript challenges from other WAFs, see WAF challenge pages.
Frequently asked questions
Does Sucuri block Googlebot by default?
Sucuri's documentation does not say it does, and its Emergency DDoS option explicitly exempts major search engines. Googlebot can still be blocked by a broad Block User Agents entry, a Geo Blocking View rule or an IP block.
What does BNP003 mean on a Sucuri Access Denied page?
It is the "Bad bot access denied" signature, which blocks bots Sucuri considers malicious by their user agent. Sucuri's examples are MJ12bot and Sogou web spider.
Can I allowlist AI crawlers by user agent in Sucuri?
Sucuri's public documentation describes allowing IP addresses and URL paths, not user agents. Allowing the operator's published IP ranges is also safer, because anyone can copy a user agent.
Why do crawlers get a blank page instead of Access Denied?
Sucuri says a blank screen is an IDS block, which lasts 20 minutes and can follow repeated 30x, 40x or 50x responses from one IP. Fix the broken URLs and redirect chains that crawlers keep hitting.
Check what crawlers get from your site
After a change, run the free check on an important page to see whether the main AI crawlers can fetch it, with no account needed. It does not connect from a crawler operator's IP addresses, so IP and geo rules may treat it differently from a real crawler, and Audit Trails remains the record. If you want firewall, robots.txt and page access reviewed together across up to 500 pages, an audit reports each problem with its evidence and a prioritized fix.
Filed under CDN and bot protection. Topics: bot-protection, waf, wordpress, troubleshooting