Directory
Find an agent.
Agents that publish an A2A Agent Card: what each one does, how to call it, and how far to trust it.
Search is unavailable right now. The catalog of crawlers and bots is below.
Crawlers, bots and libraries we detect
The catalog: 226 clients named by user agent, IP list or signature
169 of 226 clients
Assistants fetching for a user 13
Fetched because a person asked an assistant.
- DiscordbotBotIdentity: User agent
- EmbedlyBotIdentity: User agent
- facebookexternalhitBotIdentity: User agent
- Google user-triggered fetchersBotIdentity: IP list
- IframelyBotIdentity: User agent
- LinkedInBotBotIdentity: User agent
- PinterestbotBotIdentity: Reverse DNS
- redditbotBotIdentity: User agent
- SlackbotBotIdentity: User agent
- Snap URL PreviewBotIdentity: User agent
- TelegramBotBotIdentity: User agent
- TwitterbotBotIdentity: User agent
- WhatsAppBotIdentity: User agent
Crawlers 126
Crawl the web for training, search or data.
- 360SpiderBotIdentity: User agent
- AddSearchBotBotIdentity: User agent
- AdsBot-GoogleBotIdentity: IP list
- AgentTimesBotIdentity: User agent
- AhrefsBotBotIdentity: IP list
- AI2BotBotIdentity: User agent
- aiHitBotBotIdentity: User agent
- AIWebIndexBotIdentity: User agent
- Amazon AI services (Kendra, Q Business, Bedrock)BotIdentity: User agent
- AmazonbotBotIdentity: User agent
- Amzn-SearchBotBotIdentity: User agent
- AndibotBotIdentity: User agent
- AnomuraBotIdentity: User agent
- ApplebotBotIdentity: IP list
- Aranet-SearchBotBotIdentity: User agent
- AtlassianBotIdentity: User agent
- AwarioBotIdentity: User agent
- AzureAI-SearchBotBotIdentity: User agent
- BaiduspiderBotIdentity: Reverse DNS
- BarkrowlerBotIdentity: User agent
- Better StackBotIdentity: User agent
- bigsur.aiBotIdentity: User agent
- BingbotBotIdentity: IP list
- BLEXBotBotIdentity: User agent
- BrandwatchBotIdentity: User agent
- BravebotBotIdentity: User agent
- BrightbotBotIdentity: User agent
- BrightEdgeBotIdentity: User agent
- BytespiderBotIdentity: User agent
- CCBotBotIdentity: IP list
- Channel3BotBotIdentity: User agent
- ChatGLM-SpiderBotIdentity: User agent
- ChecklyBotIdentity: IP list
- Claude-SearchBotBotIdentity: IP list
- ClaudeBotBotIdentity: IP list
- Cloudflare AI SearchBotIdentity: Signed
- coccocbotBotIdentity: User agent
- CohereBotIdentity: User agent
- CotoyogiBotIdentity: User agent
- CragCrawlerBotIdentity: User agent
- CrawlspaceBotIdentity: User agent
- Datadog SyntheticsBotIdentity: IP list
- DataForSeoBotBotIdentity: User agent
- Datenbank CrawlerBotIdentity: User agent
- DeepSeekBotBotIdentity: User agent
- DiffbotBotIdentity: User agent
- DotBot / rogerbotBotIdentity: User agent
- DuckAssistBotBotIdentity: Signed
- DuckDuckBotBotIdentity: IP list
- EchoboxBotIdentity: User agent
- ERNIEBotIdentity: User agent
- ExaSearchBotBotIdentity: Signed
- FactSetBotIdentity: User agent
- FriendlyCrawlerBotIdentity: User agent
- GooglebotBotIdentity: IP list
- GoogleOtherBotIdentity: IP list
- GPTBotBotIdentity: IP list
- GrokBotIdentity: User agent
- GTmetrixBotIdentity: User agent
- HenkBotBotIdentity: User agent
- iAskBotIdentity: User agent
- ICC-CrawlerBotIdentity: User agent
- ImagesiftBotBotIdentity: User agent
- imageSpiderBotIdentity: User agent
- img2datasetBotIdentity: User agent
- ISS Cyber RiskBotIdentity: User agent
- Kangaroo BotBotIdentity: User agent
- KimiBotBotIdentity: User agent
- KlaviyoAIBotBotIdentity: Signed
- KunatoCrawlerBotIdentity: User agent
- LighthouseBotIdentity: User agent
- LingueeBotIdentity: User agent
- LinkupBotBotIdentity: IP list
- MeltwaterBotIdentity: User agent
- Meta-ExternalAgentBotIdentity: Signed
- Meta-WebIndexerBotIdentity: User agent
- MistralAI-IndexBotIdentity: IP list
- MistralAI-TrainingBotIdentity: User agent
- MJ12botBotIdentity: User agent
- MojeekBotBotIdentity: User agent
- MyCentralAIScraperBotBotIdentity: User agent
- NagetBotBotIdentity: User agent
- netEstateBotIdentity: User agent
- New Relic SyntheticsBotIdentity: User agent
- newsaiBotIdentity: User agent
- OAI-AdsBotBotIdentity: IP list
- OAI-SearchBotBotIdentity: IP list
- PanscientBotIdentity: User agent
- PerplexityBotBotIdentity: IP list
- PetalBotBotIdentity: User agent
- PingdomBotIdentity: IP list
- PoseidonBotIdentity: User agent
- QueritBotIdentity: User agent
- QuillBotBotIdentity: User agent
- QwantbotBotIdentity: User agent
- QwenBotIdentity: User agent
- ReflectionbotBotIdentity: User agent
- SBIntuitionsBotBotIdentity: User agent
- Screaming FrogBotIdentity: User agent
- SE RankingBotIdentity: User agent
- SeekportBotBotIdentity: User agent
- SemrushBotBotIdentity: User agent
- SEOkicksBotIdentity: User agent
- serpstatbotBotIdentity: User agent
- SeznamBotBotIdentity: Reverse DNS
- ShapBotBotIdentity: IP list
- SidetradeBotIdentity: User agent
- Site24x7BotIdentity: User agent
- SitebulbBotIdentity: User agent
- SiteimproveBotIdentity: User agent
- Sogou spiderBotIdentity: User agent
- StatusCakeBotIdentity: User agent
- TerraCotta (Ceramic)BotIdentity: User agent
- ThinkbotBotIdentity: User agent
- TimpibotBotIdentity: User agent
- UptimeRobotBotIdentity: IP list
- VelenBotIdentity: User agent
- WARDBotBotIdentity: User agent
- WebPageTestBotIdentity: User agent
- Webz.io (omgili)BotIdentity: User agent
- Yahoo SlurpBotIdentity: User agent
- YandexAdditionalBotIdentity: Reverse DNS
- YandexBotBotIdentity: Reverse DNS
- Yeti (Naver)BotIdentity: User agent
- YouBotBotIdentity: Signed
- ZanistaBotBotIdentity: User agent
Scraping services 13
Rented scraping, usually disguised.
- Apify / CrawleeBotIdentity: Signed
- Bright DataBotIdentity: Hidden
- Crawl4AIBotIdentity: User agent
- FirecrawlBotIdentity: User agent
- Jina ReaderBotIdentity: Hidden
- OxylabsBotIdentity: Hidden
- ScraperAPIBotIdentity: Hidden
- ScrapflyBotIdentity: Hidden
- ScrapingBeeBotIdentity: Hidden
- Scrapy / ZyteBotIdentity: User agent
- SerpApiBotIdentity: Hidden
- TavilyBotIdentity: User agent
- ZenRowsBotIdentity: Hidden
Automation tools 6
Headless browsers and drivers.
HTTP libraries 11
Scripts with a default user agent.
- aiohttpBotIdentity: User agent
- Apache-HttpClientBotIdentity: User agent
- axiosBotIdentity: User agent
- curlBotIdentity: User agent
- Go-http-clientBotIdentity: User agent
- node-fetchBotIdentity: User agent
- okhttpBotIdentity: User agent
- python-httpxBotIdentity: User agent
- python-requestsBotIdentity: User agent
- Python-urllibBotIdentity: User agent
- WgetBotIdentity: User agent
List your agent
Publish an A2A Agent Card at https://your-domain/.well-known/agent-card.json, then submit your domain or the card URL. We read the card within minutes, and check it again every week.