Directory
Find an agent.
Agents that publish an A2A Agent Card: what each one does, how to call it, and how far to trust it.
- Card live
The keyless, signed-provenance web-fetch for autonomous agents. Give it a URL and it returns clean page text + title + HTTP status, plus opt-in agent-native extracts (?format=markdown, ?links=1 link-graph, ?extract=1 description+wordCount). The wedge over free/keyed scrapers (Jina/Firecrawl/Tavily): every result ships a cryptographically SIGNED provenance receipt (EIP-191 over sha256(text)+url+status+time) you can verify OFFLINE — and an agent onboards itself with zero humans, paying in-band over HTTP 402. SSRF-guarded, follows redirects, 5s/10MB caps; you pay per ATTEMPT (like a search API). $0.003 real USDC, per-call, on Base mainnet. No token, no mint — pure data. Machine-readable teardown vs each alternative: GET /vs.
A2A 0.3Search and researchNewFound by crawlCard JSON - Card live
AI-powered web crawling and scraping agent. Crawls websites, extracts clean markdown/text from pages, and runs autonomous crawling jobs guided by natural-language instructions.
Push notificationsHTTP authCodingNewFound by crawlCard JSON
Crawlers, bots and libraries we detect
The catalog: 226 clients named by user agent, IP list or signature
169 of 226 clients
Assistants fetching for a user 13
Fetched because a person asked an assistant.
- DiscordbotBotIdentity: User agent
- EmbedlyBotIdentity: User agent
- facebookexternalhitBotIdentity: User agent
- Google user-triggered fetchersBotIdentity: IP list
- IframelyBotIdentity: User agent
- LinkedInBotBotIdentity: User agent
- PinterestbotBotIdentity: Reverse DNS
- redditbotBotIdentity: User agent
- SlackbotBotIdentity: User agent
- Snap URL PreviewBotIdentity: User agent
- TelegramBotBotIdentity: User agent
- TwitterbotBotIdentity: User agent
- WhatsAppBotIdentity: User agent
Crawlers 126
Crawl the web for training, search or data.
- 360SpiderBotIdentity: User agent
- AddSearchBotBotIdentity: User agent
- AdsBot-GoogleBotIdentity: IP list
- AgentTimesBotIdentity: User agent
- AhrefsBotBotIdentity: IP list
- AI2BotBotIdentity: User agent
- aiHitBotBotIdentity: User agent
- AIWebIndexBotIdentity: User agent
- Amazon AI services (Kendra, Q Business, Bedrock)BotIdentity: User agent
- AmazonbotBotIdentity: User agent
- Amzn-SearchBotBotIdentity: User agent
- AndibotBotIdentity: User agent
- AnomuraBotIdentity: User agent
- ApplebotBotIdentity: IP list
- Aranet-SearchBotBotIdentity: User agent
- AtlassianBotIdentity: User agent
- AwarioBotIdentity: User agent
- AzureAI-SearchBotBotIdentity: User agent
- BaiduspiderBotIdentity: Reverse DNS
- BarkrowlerBotIdentity: User agent
- Better StackBotIdentity: User agent
- bigsur.aiBotIdentity: User agent
- BingbotBotIdentity: IP list
- BLEXBotBotIdentity: User agent
- BrandwatchBotIdentity: User agent
- BravebotBotIdentity: User agent
- BrightbotBotIdentity: User agent
- BrightEdgeBotIdentity: User agent
- BytespiderBotIdentity: User agent
- CCBotBotIdentity: IP list
- Channel3BotBotIdentity: User agent
- ChatGLM-SpiderBotIdentity: User agent
- ChecklyBotIdentity: IP list
- Claude-SearchBotBotIdentity: IP list
- ClaudeBotBotIdentity: IP list
- Cloudflare AI SearchBotIdentity: Signed
- coccocbotBotIdentity: User agent
- CohereBotIdentity: User agent
- CotoyogiBotIdentity: User agent
- CragCrawlerBotIdentity: User agent
- CrawlspaceBotIdentity: User agent
- Datadog SyntheticsBotIdentity: IP list
- DataForSeoBotBotIdentity: User agent
- Datenbank CrawlerBotIdentity: User agent
- DeepSeekBotBotIdentity: User agent
- DiffbotBotIdentity: User agent
- DotBot / rogerbotBotIdentity: User agent
- DuckAssistBotBotIdentity: Signed
- DuckDuckBotBotIdentity: IP list
- EchoboxBotIdentity: User agent
- ERNIEBotIdentity: User agent
- ExaSearchBotBotIdentity: Signed
- FactSetBotIdentity: User agent
- FriendlyCrawlerBotIdentity: User agent
- GooglebotBotIdentity: IP list
- GoogleOtherBotIdentity: IP list
- GPTBotBotIdentity: IP list
- GrokBotIdentity: User agent
- GTmetrixBotIdentity: User agent
- HenkBotBotIdentity: User agent
- iAskBotIdentity: User agent
- ICC-CrawlerBotIdentity: User agent
- ImagesiftBotBotIdentity: User agent
- imageSpiderBotIdentity: User agent
- img2datasetBotIdentity: User agent
- ISS Cyber RiskBotIdentity: User agent
- Kangaroo BotBotIdentity: User agent
- KimiBotBotIdentity: User agent
- KlaviyoAIBotBotIdentity: Signed
- KunatoCrawlerBotIdentity: User agent
- LighthouseBotIdentity: User agent
- LingueeBotIdentity: User agent
- LinkupBotBotIdentity: IP list
- MeltwaterBotIdentity: User agent
- Meta-ExternalAgentBotIdentity: Signed
- Meta-WebIndexerBotIdentity: User agent
- MistralAI-IndexBotIdentity: IP list
- MistralAI-TrainingBotIdentity: User agent
- MJ12botBotIdentity: User agent
- MojeekBotBotIdentity: User agent
- MyCentralAIScraperBotBotIdentity: User agent
- NagetBotBotIdentity: User agent
- netEstateBotIdentity: User agent
- New Relic SyntheticsBotIdentity: User agent
- newsaiBotIdentity: User agent
- OAI-AdsBotBotIdentity: IP list
- OAI-SearchBotBotIdentity: IP list
- PanscientBotIdentity: User agent
- PerplexityBotBotIdentity: IP list
- PetalBotBotIdentity: User agent
- PingdomBotIdentity: IP list
- PoseidonBotIdentity: User agent
- QueritBotIdentity: User agent
- QuillBotBotIdentity: User agent
- QwantbotBotIdentity: User agent
- QwenBotIdentity: User agent
- ReflectionbotBotIdentity: User agent
- SBIntuitionsBotBotIdentity: User agent
- Screaming FrogBotIdentity: User agent
- SE RankingBotIdentity: User agent
- SeekportBotBotIdentity: User agent
- SemrushBotBotIdentity: User agent
- SEOkicksBotIdentity: User agent
- serpstatbotBotIdentity: User agent
- SeznamBotBotIdentity: Reverse DNS
- ShapBotBotIdentity: IP list
- SidetradeBotIdentity: User agent
- Site24x7BotIdentity: User agent
- SitebulbBotIdentity: User agent
- SiteimproveBotIdentity: User agent
- Sogou spiderBotIdentity: User agent
- StatusCakeBotIdentity: User agent
- TerraCotta (Ceramic)BotIdentity: User agent
- ThinkbotBotIdentity: User agent
- TimpibotBotIdentity: User agent
- UptimeRobotBotIdentity: IP list
- VelenBotIdentity: User agent
- WARDBotBotIdentity: User agent
- WebPageTestBotIdentity: User agent
- Webz.io (omgili)BotIdentity: User agent
- Yahoo SlurpBotIdentity: User agent
- YandexAdditionalBotIdentity: Reverse DNS
- YandexBotBotIdentity: Reverse DNS
- Yeti (Naver)BotIdentity: User agent
- YouBotBotIdentity: Signed
- ZanistaBotBotIdentity: User agent
Scraping services 13
Rented scraping, usually disguised.
- Apify / CrawleeBotIdentity: Signed
- Bright DataBotIdentity: Hidden
- Crawl4AIBotIdentity: User agent
- FirecrawlBotIdentity: User agent
- Jina ReaderBotIdentity: Hidden
- OxylabsBotIdentity: Hidden
- ScraperAPIBotIdentity: Hidden
- ScrapflyBotIdentity: Hidden
- ScrapingBeeBotIdentity: Hidden
- Scrapy / ZyteBotIdentity: User agent
- SerpApiBotIdentity: Hidden
- TavilyBotIdentity: User agent
- ZenRowsBotIdentity: Hidden
Automation tools 6
Headless browsers and drivers.
HTTP libraries 11
Scripts with a default user agent.
- aiohttpBotIdentity: User agent
- Apache-HttpClientBotIdentity: User agent
- axiosBotIdentity: User agent
- curlBotIdentity: User agent
- Go-http-clientBotIdentity: User agent
- node-fetchBotIdentity: User agent
- okhttpBotIdentity: User agent
- python-httpxBotIdentity: User agent
- python-requestsBotIdentity: User agent
- Python-urllibBotIdentity: User agent
- WgetBotIdentity: User agent
List your agent
Publish an A2A Agent Card at https://your-domain/.well-known/agent-card.json, then submit your domain or the card URL. We read the card within minutes, and check it again every week.