# CCBot

Bot · Crawlers · Trains AI models · operated by commoncrawl

**Identity:** Publishes its IP addresses.

## How to recognise it

- Published IP list: https://index.commoncrawl.org/ccbot.json
- Reverse DNS ends in `crawl.commoncrawl.org` (then forward-confirm)
- User agent contains `CCBot`

## Allow or block

To opt out, add to robots.txt:

```robots
User-agent: CCBot
Disallow: /
```

## With Double Agent

Sessions from CCBot are labelled `bot` with `agent_id: commoncrawl.ccbot`, and tagged in your analytics. Nothing is blocked unless you choose to. Install: https://doubleagent.so/install.md

Source: https://commoncrawl.org/ccbot