Identity: Names itself in its user agent only.
How to recognise it
- User agent contains
Diffbot,Diffbot-User
With Double Agent
Sessions from Diffbot are labelled bot with agent_id: diffbot.diffbot, and tagged in your analytics. Nothing is blocked unless you choose to. Install: https://doubleagent.so/install.md
Source: https://github.com/ai-robots-txt/ai.robots.txt