# Firecrawl

Bot · Scraping services · Collects data · operated by firecrawl

**Identity:** Names itself in its user agent only.

## How to recognise it

- User agent contains `FirecrawlAgent`

## Allow or block

To opt out, add to robots.txt:

```robots
User-agent: FirecrawlAgent
Disallow: /
```

## With Double Agent

Sessions from Firecrawl are labelled `bot` with `agent_id: firecrawl.api`, and tagged in your analytics. Nothing is blocked unless you choose to. Install: https://doubleagent.so/install.md

Source: https://github.com/mendableai/firecrawl