Skip to content

Kagibot: Kagi's crawler, its IPs and the Googlebot fallback

What Kagibot is, its user agent and four published IPs, and why it follows your Googlebot rules when robots.txt has no group that names Kagibot.

By Crawlito · 7 min read

In short

  • Kagibot is the web crawler for the Kagi search engine and identifies itself as Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot).
  • Kagi publishes four source IPs for Kagibot, each paired with a kagibot.org hostname, and documents no other way to verify a request at the time of writing.
  • If no robots.txt group names Kagibot but one names Googlebot, Kagibot follows the Googlebot group instead of the * group.
  • By Kagi's example, a file that blocks everyone except Googlebot lets Kagibot in; by the same rule, a file that blocks only Googlebot also blocks Kagibot.
  • Kagi does not document crawl-delay support, IP range files or how Kagibot treats Google-Extended, so a dedicated Kagibot group is the only predictable control.
On this page

Kagibot is the web crawler for Kagi, the paid search engine. It sends the user agent Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot), comes from four IP addresses that Kagi lists publicly, and obeys robots.txt groups that name it. The detail that surprises people is the fallback: when no group names Kagibot but one names Googlebot, Kagibot follows the Googlebot rules, not your * rules.

Everything below comes from Kagi's KagiBot page, read on 15 November 2026. The page is short and carries no date, so check it again before you hard-code anything.

What Kagibot is and how Kagi uses it

Kagi's page says one thing about purpose: Kagibot "is the web crawler for the Kagi search engine." Kagi's help page on search sources says results come from Kagi's own indexes (Teclis for the web and TinyGem for news) combined with anonymized calls to other search providers, specialized engines and vertical sources. Kagi does not say which of those indexes Kagibot feeds, or whether its pages are also used for the AI summaries Kagi shows. Treat it as a search crawler, and do not assume more than Kagi has written.

Here is what Kagi documents and what it leaves out:

TopicWhat Kagi says
User agentMozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot)
Source IPsFour addresses, each listed with a kagibot.org hostname
robots.txtStandard directives that target Kagibot are respected
FallbackFollows Googlebot directives when no rule targets Kagibot
Contactsupport@kagi.com, including reports of misbehaviour
Crawl-delayNot documented
IP range file (JSON)Not published
Training opt-out tokenNone documented

Because Kagi only describes a search crawler, it sits in the search column of the training, search and user crawler split. Blocking it means Kagi's own crawler cannot read your pages. It does not tell you whether your pages reach Kagi users through the other providers Kagi queries.

Kagibot's user agent and published IP addresses

The user agent string in Kagi's documentation:

Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot)

The robots.txt product token is Kagibot. Kagi's page writes the bot as both "KagiBot" and "Kagibot", which does not matter: RFC 9309 says crawlers must match the user-agent line case-insensitively.

Kagi lists these source addresses:

IP addressHostname listed by Kagi
216.18.205.234amd.kagibot.org
35.212.27.76donna.kagibot.org
104.254.65.50knet.kagibot.org
209.151.156.194ogi.kagibot.org

Four single addresses, not ranges. That makes Kagibot easier to allowlist than most crawlers, but it also means a new address Kagi adds will not pass your firewall until you update it. Kagi does not publish a machine-readable file you could fetch on a schedule, unlike Apple's applebot.json or the lists described in AI crawler IP ranges for firewall allowlists.

How to verify a request really is Kagibot

The usual method for Googlebot or Applebot is reverse DNS, then forward DNS: look up the hostname for the IP, check the domain, then confirm the hostname resolves back to the same IP. That method does not work for Kagibot at the time of writing.

When we checked on 15 November 2026, each kagibot.org hostname resolved forward to the IP Kagi lists, but none of the four IPs had a reverse record pointing back to kagibot.org. Two returned their hosting providers' generic names, and for the other two the reverse lookup returned no record. Kagi's page does not tell you to use reverse DNS, so this is not a fault, but a script that requires a matching PTR record will reject real Kagibot traffic.

Verify against the list instead. This counts Kagibot requests in an access log and splits them by whether the client IP is one Kagi publishes:

bash
grep -i "kagibot" access.log \
  | awk '{print $1}' \
  | sort | uniq -c \
  | awk 'BEGIN{ok["216.18.205.234"];ok["35.212.27.76"];ok["104.254.65.50"];ok["209.151.156.194"]}
         {print $1, $2, ($2 in ok ? "listed" : "NOT LISTED")}'

The first field assumes the common combined log format; behind a CDN or load balancer, log the real client IP or this compares the wrong address. To confirm the hostnames still map to the listed IPs, resolve them forward:

bash
for h in amd donna knet ogi; do dig +short "$h.kagibot.org" A; done

Anything that claims to be Kagibot from an unlisted IP is either an address Kagi has not yet documented or someone borrowing the name. The general method, and how it differs per operator, is in how to verify a real AI crawler. If you see unlisted IPs in volume and want to know whether they are Kagi's, support@kagi.com is the contact Kagi's page gives for questions.

The Googlebot fallback: what Kagibot reads in your robots.txt

Kagi's rule, in its own words: "If there is no rule targeting kagibot, but there is a rule targeting Googlebot, then Kagibot will follow the Googlebot directives."

That changes the normal RFC 9309 order. A standard crawler uses its own group if one exists, otherwise *, as explained in how robots.txt groups work. Kagibot inserts a step in between:

  1. A group that names Kagibot.
  2. If none, a group that names Googlebot.
  3. If neither, the * group.

Kagi's own example makes the effect plain. In this file, every crawler is blocked except Googlebot, which may crawl everything outside /private/:

User-Agent: *
Disallow: /

User-Agent: Googlebot
Allow: /
Disallow: /private/

Kagi says Kagibot fetches all pages except those under /private/ here. So a "Google only" robots.txt is not Google only.

The reverse case follows from the same rule, though Kagi's page only shows the example above. A file that shuts out Google and allows everyone else blocks Kagibot too:

User-agent: Googlebot
Disallow: /

User-agent: *
Allow: /

Kagibot finds no group of its own, finds a Googlebot group, and reads Disallow: /.

What the fallback does not tell you

Kagi's page is one paragraph on this, so several edge cases are undocumented at the time of writing:

  • Whether a group naming only another Google token, such as Googlebot-Image or Googlebot-News, counts as "a rule targeting Googlebot".
  • Whether Kagibot pays any attention to Google-Extended, which is a Google training token and not a crawler.
  • Whether Kagibot honours Crawl-delay, or how it reacts to slow responses and errors.

Do not rely on any of these. If the answer matters to you, name Kagibot directly.

How Kagibot's fallback compares with Applebot's

Apple documents the same kind of fallback. Its About Applebot page says: "If robots instructions don't mention Applebot but mention Googlebot, the Apple robot will follow Googlebot instructions." The two operators differ in what else they give you:

KagibotApplebot
Falls back to Googlebot groupYes, when no group names KagibotYes, when robots.txt does not mention Applebot
Separate training opt-out tokenNone documentedApplebot-Extended
VerificationFour listed IPs with hostnamesReverse and forward DNS under applebot.apple.com, plus applebot.json
Crawl-delayNot documentedApple says it is not followed

The practical lesson is the same for both. Any time you edit a Googlebot group, you may be editing the rules for two other crawlers as well. The Applebot and Applebot-Extended guide covers Apple's side in detail.

How to write a Kagibot group

A named group ends the fallback. Kagi's example of a group that targets it:

User-Agent: Kagibot
Allow: /
Disallow: /private/

Under RFC 9309, a named group is the only group that crawler reads, so it does not inherit anything from * or from Googlebot. Copy in every rule you still want it to obey. To keep Kagibot on the same policy as Google while making that explicit, stack the tokens in one group:

User-agent: Googlebot
User-agent: Kagibot
User-agent: Applebot
Disallow: /admin/
Disallow: /cart/

User-agent: *
Disallow: /

To block Kagibot without touching Google:

User-agent: Kagibot
Disallow: /

User-agent: Googlebot
Allow: /

Then check the result:

  1. Fetch the live file with curl -s https://www.example.com/robots.txt, since CMS plugins and CDNs can change what is served.
  2. Search it for kagibot and googlebot, case-insensitively. If Kagibot is absent and Googlebot is present, the Googlebot group is Kagibot's policy.
  3. Make sure your firewall or bot protection does not block the four listed IPs if you want Kagibot to crawl. robots.txt cannot help a crawler that never gets past the edge.
  4. Watch your logs for Kagibot requests to paths you disallowed. Kagi says standard directives are respected, so repeated hits on a disallowed path are worth reporting to Kagi.

Frequently asked questions

Is Kagibot an AI training crawler?

Kagi describes Kagibot only as the web crawler for the Kagi search engine. It does not document any training use or a separate training opt-out token.

Why does Kagibot crawl pages I only allowed for Googlebot?

Kagi says that when no robots.txt rule targets Kagibot but one targets Googlebot, Kagibot follows the Googlebot directives. Add a User-agent: Kagibot group if you want a different policy.

How do I check that a Kagibot request is genuine?

Compare the client IP with the four addresses on Kagi's bot page. When we checked on 15 November 2026, none of the IPs reverse-resolved to kagibot.org, so a reverse DNS test that expects that domain rejects real requests.

Does Kagibot obey Crawl-delay?

Kagi's documentation does not say. If load is a problem, rate limit the listed IPs at your server or contact support@kagi.com.

Check your robots.txt after a Googlebot edit

The fallback means one Googlebot edit can change access for Google, Apple and Kagi at once. After any robots.txt change, run the free check on an important page to see how your live file treats Googlebot, Applebot and the AI crawlers it covers; unless you name Kagibot, its policy is the Googlebot result. The check covers one page and needs no account.

Filed under AI crawlers. Topics: ai-crawlers, robots-txt, googlebot, server-logs