Directory Crawlers

ClaudeBot Bot

Trains AI models, operated by anthropic. Crawl the web for training, search or data.

Operator
anthropic
Kind
Crawlers
Does
Trains AI models
Traffic
Its operator’s own
robots.txt
ClaudeBot

Identity

Publishes its IP addresses

  1. SignedSigns every request (Web Bot Auth): cryptographic proof.
  2. IP listPublishes the IP addresses it comes from.
  3. Reverse DNSIts IP addresses resolve to the operator’s domain.
  4. User agentNames itself in its user agent, which anyone can copy.
  5. HiddenLooks like a person’s browser: detected by behaviour.

How to recognise it

Details

ID
anthropic.claudebot
Category
Bot
Operator
anthropic
Agent Card
Not published: this agent doesn’t identify itself with an Agent Card.
First seen
2026-09-28

Registrations

Reputation

Computed 2026-11-13

—No score yet
Confidence8%

A score shows from 30%

  • Identity40% of the score19 / 100

    0 verified sources

    A verified Agent Card adds 25 points, 40 if it is signed.

  • Reviews35% of the score–

    No reviews yet

    From the ERC-8004 reputation registry, weighted by who wrote them.

  • Observed25% of the score–

    Not enough sightings yet

    Counts once it is seen on 10 or more Double Agent sites.

Change history

No card history: this agent doesn’t publish an Agent Card.

Allow or block

To opt out, add this to your robots.txt.

Double Agent keeps labelling it either way, so you can see whether it listens.

User-agent: ClaudeBot
Disallow: /

With Double Agent

Sessions from ClaudeBot are labelled bot with agent_id: anthropic.claudebot, and tagged in your analytics. Nothing is blocked unless you choose to.

Install Double Agent

Source: https://support.claude.com/en/articles/8896518