AI search crawler

ExaSearchBot

ExaSearchBot is the crawler Exa uses to discover and index pages on the public web, so they can be found, retrieved and cited through Exa's search engine and API. Exa says every ExaSearchBot request is signed with HTTP Message Signatures under the Web Bot Auth scheme.

Sources: Exa's own documentation and the Sill agent registry · checked 17 Nov 2026

Company
Exa
Acts for
Exa, for its search engine and search API
Follows robots.txt
Yes
How to verify
Request signature

robots.txt

Exa says ExaSearchBot respects robots.txt. It looks for a group addressed to ExaSearchBot; without one it follows the rules you set for major search engines, otherwise the * group. Exa says robots.txt controls crawling, not indexing, so use noindex to remove pages already indexed.

User-agent: ExaSearchBot
Allow: /

How to check it is really ExaSearchBot

  • Request signature: Every request is signed (RFC 9421, Web Bot Auth) with Signature, Signature-Input and Signature-Agent headers; Ed25519 keys at https://crawler.exa.ai/.well-known/http-message-signatures-directory

How verification works →

Sill Agent Intelligence

What it does on websites PROFILE IN PROGRESS

Sill Agent Intelligence is building ExaSearchBot's behaviour profile from verified traffic across the Sill network: reach, identity checks, activity trend and the pages it targets. It appears here automatically when ready.

Other Exa agents

Read more on Signals

Questions about ExaSearchBot

Should I block ExaSearchBot?

Only if you do not want your pages in Exa's index. Exa says the index lets users find, retrieve and cite your content and send people back to your site.

Can I verify it by IP address?

Exa's crawler page lists no IP ranges for it. Verify the request signature against Exa's public key directory instead; Exa says this holds regardless of the request's IP address.

Sources

  1. Exa: Exa Search Crawler · checked 17 Nov 2026
  2. Exa: ExaSearchBot public key directory · checked 17 Nov 2026

Official documentation: https://crawler.exa.ai/ · Spotted an error? Tell us

See which AI agents visit your site, and decide what they may do.

Scan your site free