Directory Crawlers

Amazonbot Bot

Trains AI models. Operated by amazon.

Identity: Names itself in its user agent only.

How to recognise it

  • User agent contains Amazonbot

Allow or block

To opt out, add to robots.txt:

User-agent: Amazonbot
Disallow: /

With Double Agent

Sessions from Amazonbot are labelled bot with agent_id: amazon.amazonbot, and tagged in your analytics. Nothing is blocked unless you choose to. Install: https://doubleagent.so/install.md

Source: https://developer.amazon.com/amazonbot ยท Operator docs: https://developer.amazon.com/amazonbot