Kagibot: Kagi's crawler, its IPs and the Googlebot fallback
What Kagibot is, its user agent and four published IPs, and why it follows your Googlebot rules when robots.txt has no group that names Kagibot.
By Crawlito · 7 min read
In short
- Kagibot is the web crawler for the Kagi search engine and identifies itself as Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot).
- Kagi publishes four source IPs for Kagibot, each paired with a kagibot.org hostname, and documents no other way to verify a request at the time of writing.
- If no robots.txt group names Kagibot but one names Googlebot, Kagibot follows the Googlebot group instead of the * group.
- By Kagi's example, a file that blocks everyone except Googlebot lets Kagibot in; by the same rule, a file that blocks only Googlebot also blocks Kagibot.
- Kagi does not document crawl-delay support, IP range files or how Kagibot treats Google-Extended, so a dedicated Kagibot group is the only predictable control.
On this page
- What Kagibot is and how Kagi uses it
- Kagibot's user agent and published IP addresses
- How to verify a request really is Kagibot
- The Googlebot fallback: what Kagibot reads in your robots.txt
- How Kagibot's fallback compares with Applebot's
- How to write a Kagibot group
- Frequently asked questions
- Check your robots.txt after a Googlebot edit
Kagibot is the web crawler for Kagi, the paid search engine. It sends the user agent Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot), comes from four IP addresses that Kagi lists publicly, and obeys robots.txt groups that name it. The detail that surprises people is the fallback: when no group names Kagibot but one names Googlebot, Kagibot follows the Googlebot rules, not your * rules.
Everything below comes from Kagi's KagiBot page, read on 15 November 2026. The page is short and carries no date, so check it again before you hard-code anything.
What Kagibot is and how Kagi uses it
Kagi's page says one thing about purpose: Kagibot "is the web crawler for the Kagi search engine." Kagi's help page on search sources says results come from Kagi's own indexes (Teclis for the web and TinyGem for news) combined with anonymized calls to other search providers, specialized engines and vertical sources. Kagi does not say which of those indexes Kagibot feeds, or whether its pages are also used for the AI summaries Kagi shows. Treat it as a search crawler, and do not assume more than Kagi has written.
Here is what Kagi documents and what it leaves out:
| Topic | What Kagi says |
|---|---|
| User agent | Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot) |
| Source IPs | Four addresses, each listed with a kagibot.org hostname |
| robots.txt | Standard directives that target Kagibot are respected |
| Fallback | Follows Googlebot directives when no rule targets Kagibot |
| Contact | support@kagi.com, including reports of misbehaviour |
| Crawl-delay | Not documented |
| IP range file (JSON) | Not published |
| Training opt-out token | None documented |
Because Kagi only describes a search crawler, it sits in the search column of the training, search and user crawler split. Blocking it means Kagi's own crawler cannot read your pages. It does not tell you whether your pages reach Kagi users through the other providers Kagi queries.
Kagibot's user agent and published IP addresses
The user agent string in Kagi's documentation:
Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot)The robots.txt product token is Kagibot. Kagi's page writes the bot as both "KagiBot" and "Kagibot", which does not matter: RFC 9309 says crawlers must match the user-agent line case-insensitively.
Kagi lists these source addresses:
| IP address | Hostname listed by Kagi |
|---|---|
216.18.205.234 | amd.kagibot.org |
35.212.27.76 | donna.kagibot.org |
104.254.65.50 | knet.kagibot.org |
209.151.156.194 | ogi.kagibot.org |
Four single addresses, not ranges. That makes Kagibot easier to allowlist than most crawlers, but it also means a new address Kagi adds will not pass your firewall until you update it. Kagi does not publish a machine-readable file you could fetch on a schedule, unlike Apple's applebot.json or the lists described in AI crawler IP ranges for firewall allowlists.
How to verify a request really is Kagibot
The usual method for Googlebot or Applebot is reverse DNS, then forward DNS: look up the hostname for the IP, check the domain, then confirm the hostname resolves back to the same IP. That method does not work for Kagibot at the time of writing.
When we checked on 15 November 2026, each kagibot.org hostname resolved forward to the IP Kagi lists, but none of the four IPs had a reverse record pointing back to kagibot.org. Two returned their hosting providers' generic names, and for the other two the reverse lookup returned no record. Kagi's page does not tell you to use reverse DNS, so this is not a fault, but a script that requires a matching PTR record will reject real Kagibot traffic.
Verify against the list instead. This counts Kagibot requests in an access log and splits them by whether the client IP is one Kagi publishes:
grep -i "kagibot" access.log \
| awk '{print $1}' \
| sort | uniq -c \
| awk 'BEGIN{ok["216.18.205.234"];ok["35.212.27.76"];ok["104.254.65.50"];ok["209.151.156.194"]}
{print $1, $2, ($2 in ok ? "listed" : "NOT LISTED")}'The first field assumes the common combined log format; behind a CDN or load balancer, log the real client IP or this compares the wrong address. To confirm the hostnames still map to the listed IPs, resolve them forward:
for h in amd donna knet ogi; do dig +short "$h.kagibot.org" A; doneAnything that claims to be Kagibot from an unlisted IP is either an address Kagi has not yet documented or someone borrowing the name. The general method, and how it differs per operator, is in how to verify a real AI crawler. If you see unlisted IPs in volume and want to know whether they are Kagi's, support@kagi.com is the contact Kagi's page gives for questions.
The Googlebot fallback: what Kagibot reads in your robots.txt
Kagi's rule, in its own words: "If there is no rule targeting kagibot, but there is a rule targeting Googlebot, then Kagibot will follow the Googlebot directives."
That changes the normal RFC 9309 order. A standard crawler uses its own group if one exists, otherwise *, as explained in how robots.txt groups work. Kagibot inserts a step in between:
- A group that names
Kagibot. - If none, a group that names
Googlebot. - If neither, the
*group.
Kagi's own example makes the effect plain. In this file, every crawler is blocked except Googlebot, which may crawl everything outside /private/:
User-Agent: *
Disallow: /
User-Agent: Googlebot
Allow: /
Disallow: /private/Kagi says Kagibot fetches all pages except those under /private/ here. So a "Google only" robots.txt is not Google only.
The reverse case follows from the same rule, though Kagi's page only shows the example above. A file that shuts out Google and allows everyone else blocks Kagibot too:
User-agent: Googlebot
Disallow: /
User-agent: *
Allow: /Kagibot finds no group of its own, finds a Googlebot group, and reads Disallow: /.
What the fallback does not tell you
Kagi's page is one paragraph on this, so several edge cases are undocumented at the time of writing:
- Whether a group naming only another Google token, such as
Googlebot-ImageorGooglebot-News, counts as "a rule targeting Googlebot". - Whether Kagibot pays any attention to
Google-Extended, which is a Google training token and not a crawler. - Whether Kagibot honours
Crawl-delay, or how it reacts to slow responses and errors.
Do not rely on any of these. If the answer matters to you, name Kagibot directly.
How Kagibot's fallback compares with Applebot's
Apple documents the same kind of fallback. Its About Applebot page says: "If robots instructions don't mention Applebot but mention Googlebot, the Apple robot will follow Googlebot instructions." The two operators differ in what else they give you:
| Kagibot | Applebot | |
|---|---|---|
| Falls back to Googlebot group | Yes, when no group names Kagibot | Yes, when robots.txt does not mention Applebot |
| Separate training opt-out token | None documented | Applebot-Extended |
| Verification | Four listed IPs with hostnames | Reverse and forward DNS under applebot.apple.com, plus applebot.json |
| Crawl-delay | Not documented | Apple says it is not followed |
The practical lesson is the same for both. Any time you edit a Googlebot group, you may be editing the rules for two other crawlers as well. The Applebot and Applebot-Extended guide covers Apple's side in detail.
How to write a Kagibot group
A named group ends the fallback. Kagi's example of a group that targets it:
User-Agent: Kagibot
Allow: /
Disallow: /private/Under RFC 9309, a named group is the only group that crawler reads, so it does not inherit anything from * or from Googlebot. Copy in every rule you still want it to obey. To keep Kagibot on the same policy as Google while making that explicit, stack the tokens in one group:
User-agent: Googlebot
User-agent: Kagibot
User-agent: Applebot
Disallow: /admin/
Disallow: /cart/
User-agent: *
Disallow: /To block Kagibot without touching Google:
User-agent: Kagibot
Disallow: /
User-agent: Googlebot
Allow: /Then check the result:
- Fetch the live file with
curl -s https://www.example.com/robots.txt, since CMS plugins and CDNs can change what is served. - Search it for
kagibotandgooglebot, case-insensitively. If Kagibot is absent and Googlebot is present, the Googlebot group is Kagibot's policy. - Make sure your firewall or bot protection does not block the four listed IPs if you want Kagibot to crawl. robots.txt cannot help a crawler that never gets past the edge.
- Watch your logs for Kagibot requests to paths you disallowed. Kagi says standard directives are respected, so repeated hits on a disallowed path are worth reporting to Kagi.
Frequently asked questions
Is Kagibot an AI training crawler?
Kagi describes Kagibot only as the web crawler for the Kagi search engine. It does not document any training use or a separate training opt-out token.
Why does Kagibot crawl pages I only allowed for Googlebot?
Kagi says that when no robots.txt rule targets Kagibot but one targets Googlebot, Kagibot follows the Googlebot directives. Add a User-agent: Kagibot group if you want a different policy.
How do I check that a Kagibot request is genuine?
Compare the client IP with the four addresses on Kagi's bot page. When we checked on 15 November 2026, none of the IPs reverse-resolved to kagibot.org, so a reverse DNS test that expects that domain rejects real requests.
Does Kagibot obey Crawl-delay?
Kagi's documentation does not say. If load is a problem, rate limit the listed IPs at your server or contact support@kagi.com.
Check your robots.txt after a Googlebot edit
The fallback means one Googlebot edit can change access for Google, Apple and Kagi at once. After any robots.txt change, run the free check on an important page to see how your live file treats Googlebot, Applebot and the AI crawlers it covers; unless you name Kagibot, its policy is the Googlebot result. The check covers one page and needs no account.
Filed under AI crawlers. Topics: ai-crawlers, robots-txt, googlebot, server-logs