Browserbase
Browserbase and Anchor Browser: the cloud browsers behind many AI agents
Browserbase and Anchor Browser run cloud browsers for AI agents. What their traffic is, how to recognise signed sessions, and what store owners should do.
One AI agent at a time: who runs it, what it does on your site, how to verify it.
Browserbase
Browserbase and Anchor Browser run cloud browsers for AI agents. What their traffic is, how to recognise signed sessions, and what store owners should do.
Google-GeminiNotebook
Google-GeminiNotebook, formerly Google-NotebookLM, fetches pages people add as sources in Gemini Notebook. Here's what it does, robots.txt and verification.
Buy for Me
How Amazon's Buy for Me and Alexa for Shopping buy from third-party stores, how Amazon's bots identify themselves, and how merchants join or opt out.
Microsoft Copilot
How Microsoft Copilot reads your store through Bing and feeds, how Browse with Copilot and Copilot Tasks act on websites, and what that means for your logs.
ExaSearchBot
What ExaSearchBot is, what it looks like in your logs, how to verify its Web Bot Auth signatures, and how to allow or block it with robots.txt.
ChatGPT agent
What ChatGPT agent does on your website, how it identifies itself with signed requests, and how it differs from GPTBot, OAI-SearchBot and ChatGPT-User.
Claude in Chrome
What Claude in Chrome does on your site, whether it can buy from your store, how it identifies itself, and how it differs from ClaudeBot and Claude-User.
Perplexity Comet
What Perplexity Comet does on your website, why it is hard to tell apart from Chrome, how it differs from PerplexityBot, and where the Amazon case stands.
Google-Agent
What the Google-Agent user agent in your logs means, why it ignores robots.txt, and how to verify it with Google's IP list and Web Bot Auth.
Meta's AI crawlers
Meta's AI crawlers explained: what Meta-ExternalAgent, Meta-ExternalFetcher and Meta-WebIndexer do, how robots.txt applies, and which ones to allow.