Our scanner.
Sill-Readiness checks whether a site is ready for AI agents. This page says what the scanner is, who starts a scan, what it reads, and how to block it.
One User-Agent.
Every scan request carries this string. The URL after the plus points at this page.
Sill-Readiness/1 (+https://sill.so/bot)
A scan runs only for a specific site.
The scanner does not crawl the web or follow links from site to site. It checks one site at a time, for one of these reasons.
- The site owner, or a Sill customer, from the dashboard, at signup, as a scheduled re-check of an installed site, or as part of an agency's domain import.
- Anyone who types a domain into the scan form at sill.so.
A short list of public files.
Every request is HTTPS on port 443 and GET only, with one small POST noted below. The scanner follows redirects only within the same site.
- The homepageOne GET to confirm the site is reachable and read its structured markup.
- /robots.txtFetched once per scan, both as a signal and to apply its rules to the files below.
- A sitemap, and a few sitemaps listed in itThe common sitemap paths; at most five linked sitemaps.
- /llms.txt and /llms-full.txtThe two conventional llms.txt locations.
- /.well-known/agent-card.jsonThe A2A agent card.
- /.well-known/ai-catalog.jsonThe ARD agent-ready catalog.
- /.well-known/api-catalogThe RFC 9727 api-catalog resource.
- /.well-known/ucpThe UCP business profile.
- The MCP well-known paths/.well-known/mcp/server-card.json and the earlier /.well-known/mcp.json.
- /agentThe HTML agent gateway, when present.
- A few URLs those files point toFor example an MCP server URL named in the agent card, or a sitemap URL listed in robots.txt.
- A few policy pages, after a site owner signs upWhen a merchant adds their site to Sill, the scanner reads common policy pages — /privacy, /terms, /returns, /shipping, and the Shopify /policies/* paths — to prefill the onboarding form. Only after that owner started the process.
If an agent card or similar file names an MCP server, the scanner sends one small JSON-RPC tools/list request to that server to check that it answers. No other POSTs are issued.
Bounded on both ends.
- Per scan, to the siteAt most 30 requests to the site being scanned.
- Per scan, to other hostsAt most 4 requests to other hosts the site's own files point to.
- Per visitorA site can be scanned at most 10 times every 5 minutes from one visitor.
- Per siteAcross all visitors, a site can be scanned at most 20 times every 5 minutes.
The scanner obeys robots.txt.
The scanner reads the rules under User-agent: Sill-Readiness first. If none are set, it reads the rules under User-agent: *.
To block the scanner from reading anything, add:
User-agent: Sill-Readiness Disallow: /
A path blocked by robots.txt is not fetched. In the owner's readiness report, that path shows up as blocked by robots.txt.
Requests to your site are cryptographically signed.
The scanner proves it is Sill with a signature on each request it sends to the site being checked, following the IETF Web Bot Auth profile (RFC 9421 HTTP Message Signatures, tag web-bot-auth). A verifier can confirm the signature against a published Ed25519 public-key directory.
- Signature-Agent
"https://edge.sill.so/agents/sill-readiness" - Covered components
"@authority"and"signature-agent". Signature lifetimecreated+ 60 seconds. - AlgorithmEd25519. 128-bit random nonce on each request.
- Public-key directory
https://edge.sill.so/agents/sill-readiness/.well-known/http-message-signatures-directory. Served with content typeapplication/http-message-signatures-directory+json, cached one minute at the edge.
Follow-up requests to a different domain the site points to, such as a third-party MCP server, are sent without a signature, so the signature is only presented to the site that was scanned. If signing is unavailable for a request, the scanner sends it without signature headers rather than failing a scan the site owner asked for. The User-Agent is unchanged either way.
Reach a person.
Write to [email protected] for anything about the scanner, including a request to remove a domain from a scheduled re-check.