Trust & security

Verifying AI bots by IP range and reverse DNS: a 2026 reference

Listen · 12 min

Narrated by an AI voice. Code examples are on the page.

Short answer

Most AI bot operators publish the IP addresses their bots use, and a few also support a reverse DNS check. OpenAI, Anthropic, Perplexity, Google, Apple, Amazon, Common Crawl, DuckDuckGo and Microsoft's Bing publish IP lists; Google, Bing, Apple and Common Crawl document reverse DNS. Meta's crawler page gives no list or DNS method. Match a request's IP against the list for the exact bot it claims to be.

Key takeaways

  • Check the IP against the list for the exact bot named.
  • Reverse DNS works only where the operator documents a hostname.
  • Meta's crawler page publishes no IP list or DNS method.

Why verify AI bots by IP range or reverse DNS?

Because the bot's name proves nothing. A user agent is text the sender chooses, and Google says plainly: "The HTTP user agent string can be spoofed." The IP address a request comes from is much harder to fake, and operators publish where their bots connect from so you can check.

There are two network checks:

  • IP list match. The operator publishes a file of IP ranges in CIDR form. If the request's IP falls inside a range for the bot it claims to be, it passes.
  • Reverse DNS. You look up the hostname for the IP, check it ends in the operator's domain, then look that hostname up again to confirm it points back to the same IP. See reverse DNS verification for the steps.

A third method, a cryptographic signature, is covered in our guide to Web Bot Auth. This page is the reference for the network checks.

Which AI bot operators publish IP ranges or reverse DNS?

Here is what each operator's own documentation says at the time of writing. "Not documented" means the operator page does not describe that method.

Operator Bots IP list Reverse DNS hostname
OpenAI GPTBot openai.com/gptbot.json Not documented
OpenAI OAI-SearchBot openai.com/searchbot.json Not documented
OpenAI ChatGPT-User openai.com/chatgpt-user.json Not documented
OpenAI OAI-AdsBot openai.com/adsbot.json Not documented
Anthropic ClaudeBot, Claude-User, Claude-SearchBot claude.com/crawling/bots.json Not documented
Anthropic Claude-Agent claude.com/.well-known/sail Not documented
Google Googlebot and other common crawlers common-crawlers.json googlebot.com
Google Google-Agent user-triggered-agents.json gae.googleusercontent.com or google.com (user-triggered fetcher masks)
Perplexity PerplexityBot perplexity.ai/perplexitybot.json Not documented
Perplexity Perplexity-User perplexity.ai/perplexity-user.json Not documented
Microsoft Bingbot bing.com/toolbox/bingbot.json search.msn.com
Apple Applebot search.developer.apple.com/applebot.json applebot.apple.com
Amazon Amazonbot, Amzn-SearchBot, Amzn-User One web page per bot on developer.amazon.com/amazonbot Not documented
Meta Meta-ExternalAgent, Meta-ExternalFetcher None published Not documented
Common Crawl CCBot index.commoncrawl.org/ccbot.json crawl.commoncrawl.org
DuckDuckGo DuckAssistBot duckduckgo.com/duckassistbot.json Not documented

Google's files all sit under https://developers.google.com/static/crawling/ipranges/. Most of the JSON files above share one layout: a creationTime and a prefixes list of ipv4Prefix and ipv6Prefix entries. Anthropic's Claude-Agent file uses a different format, and Amazon lists its addresses on web pages rather than in JSON.

How do the operator-specific checks differ?

The idea is the same everywhere, but each operator adds its own detail. These notes come from each operator's documentation.

OpenAI publishes one file per bot. Check the file for the bot the request names, not any OpenAI file.

Anthropic publishes one shared file for ClaudeBot, Claude-User and Claude-SearchBot, and says an IP on that list "indicates that the crawler is coming from Anthropic." Its hosted browser agent, Claude-Agent, has a separate file at claude.com/.well-known/sail, with addresses "listed under Claude-Agent." Anthropic says the format follows an IETF draft and "is not yet a standard, so it may evolve over time."

Google documents both methods and splits its bots into groups, each with its own file and hostname pattern. Common crawlers such as Googlebot resolve to crawl-***-***-***-***.googlebot.com or geo-crawl-***-***-***-***.geo.googlebot.com. User-triggered fetchers resolve to gae.googleusercontent.com or google-proxy-***-***-***-***.google.com, with their own files. A request that claims to be Googlebot but resolves to a fetcher hostname is not Googlebot.

Perplexity recommends you "combine both User-Agent string matching and IP address verification for enhanced security."

Microsoft (Bing) documents reverse DNS: the hostname should "end with search.msn.com," then a forward lookup should "confirm that it resolves back to the same IP address." Bing's 2012 guide says Bing does not publish an IP list. Bing now hosts bingbot.json, which loaded when we checked, with a creationTime of January 2024. Use the DNS check as your main method for Bingbot.

Apple says Applebot traffic "is generally identified by using reverse DNS in the *.applebot.apple.com domain," and also publishes applebot.json to match against.

Amazon lists separate addresses for Amazonbot, Amzn-SearchBot and Amzn-User, each on its own page. It does not document reverse DNS.

Common Crawl says CCBot "is now run on dedicated IP address ranges with reverse DNS (except over IPv6 where reverse DNS is not yet supported.)" Its JSON file also lists the ranges.

DuckDuckGo links to duckassistbot.json from its DuckAssistBot page, without further verification steps.

Which AI bots have no IP range to check?

Some bots publish no network check, and some never visit at all.

  • Meta. Meta's crawler page tells you to allow-list "either the user agent strings or the IP addresses (more secure) used by the crawler," but it gives no IP list, file or DNS hostname. We could not verify Meta's crawlers by network from its documentation. For what each Meta bot does, see Meta's AI crawlers explained.
  • ChatGPT agent. OpenAI's guide for ChatGPT agent describes signed requests with a Signature-Agent header of "https://chatgpt.com". It does not give an IP list for it. OpenAI says Akamai, Cloudflare, HUMAN and Vercel verify these signatures automatically.
  • Google-Extended and Applebot-Extended. These are robots.txt tokens, not crawlers. Google says Google-Extended "doesn't have a separate HTTP request user agent string," and Apple says "Applebot-Extended does not crawl webpages." There is nothing to verify by IP.

How do you run the checks in practice?

Automate both checks and run them only when a request claims to be a known bot.

  1. Read the claimed bot from the user agent. If it names no bot you track, treat it as a normal visitor.
  2. Pick the matching method. Use the table above. Prefer the operator's list for that exact bot.
  3. Match the IP. Check whether it falls inside any CIDR range in the file.
  4. Or run reverse DNS where the operator documents a hostname:
$ host 18.97.14.84
84.14.97.18.in-addr.arpa domain name pointer 18-97-14-84.crawl.commoncrawl.org.
$ host 18-97-14-84.crawl.commoncrawl.org
18-97-14-84.crawl.commoncrawl.org has address 18.97.14.84

That example is from Common Crawl's own page. The hostname is under crawl.commoncrawl.org and points back to the same IP, so it passes.

  1. Refresh the lists on a schedule. Anthropic recommends updating rules from its Claude-Agent file automatically rather than hardcoding addresses, and the same habit suits every list.
  2. Label failures neutrally. A request that claims to be a bot but fails the check is "claims GPTBot, failed verification," not GPTBot. Never give it the access the real bot gets. Our guide to fake GPTBot, ClaudeBot and Googlebot covers what to do next.

An IP check tells you which network a request came from, not who the person behind an agent is. For how network checks fit with signatures and your own rules, see the AI agent identity guide.

How Sill helps

Sill shows you which AI agents and bots visit your site and the pages they read. It verifies signed agent requests (Web Bot Auth) on Cloudflare and WordPress installs, for agents that sign. To see how your site looks to AI agents today, run the free readiness scan, no account needed.

Questions people ask

How often should I refresh AI bot IP lists?

Operators add and remove ranges without notice, and Anthropic says its Claude-Agent addresses may change. Fetch the files automatically, daily is a sensible default, rather than copying addresses into a config by hand.

Can I verify ChatGPT agent by IP address?

OpenAI's guide for ChatGPT agent describes signed requests, not an IP list. It says to verify the signature against keys at chatgpt.com, and that Akamai, Cloudflare, HUMAN and Vercel verify these signatures automatically.

Why does Google-Extended have no IP range?

Because it never visits your site. Google says Google-Extended has no separate user agent string and that crawling is done with existing Google user agents. It is only a robots.txt control token. Applebot-Extended works the same way.

Is a request from an AI company's IP range always that company's bot?

It is strong evidence, which is why operators publish the lists. Match against the list for the specific bot named in the user agent, though, not any list from that company.

Sources

  1. OpenAI: Overview of OpenAI crawlers · checked 17 Nov 2026
  2. OpenAI: Help ensure cloud browser access to your site · checked 17 Nov 2026
  3. Anthropic: Does Anthropic crawl data from the web? · checked 17 Nov 2026
  4. Claude Help Center: How do Anthropic-hosted browsers identify themselves? · checked 17 Nov 2026
  5. Google: Verify requests from Google crawlers and fetchers · checked 17 Nov 2026
  6. Google: Google's common crawlers · checked 17 Nov 2026
  7. Google: Google's user-triggered fetchers · checked 17 Nov 2026
  8. Google: Allow Google-Agent traffic · checked 17 Nov 2026
  9. Perplexity: Perplexity crawlers · checked 17 Nov 2026
  10. Bing Webmaster Blog: How to verify that Bingbot is Bingbot · checked 17 Nov 2026
  11. Bing: bingbot.json · checked 17 Nov 2026
  12. Apple: About Applebot · checked 17 Nov 2026
  13. Amazon Developer: Amazonbot · checked 17 Nov 2026
  14. Meta: Web crawlers · checked 17 Nov 2026
  15. Common Crawl: CCBot · checked 17 Nov 2026
  16. DuckDuckGo: DuckAssistBot · checked 17 Nov 2026

Written by The Sill Signals Team. Last reviewed 17 Nov 2026.

Share this articleLinkedInXBlueskyEmail

Keep reading

See which AI agents visit your site, and decide what they may do.

Scan your site free