Directory Crawlers

img2dataset Bot

Collects data. Operated by laion.

Identity: Names itself in its user agent only.

How to recognise it

  • User agent contains img2dataset, LAIONDownloader, laion-huggingface-processor

With Double Agent

Sessions from img2dataset are labelled bot with agent_id: laion.img2dataset, and tagged in your analytics. Nothing is blocked unless you choose to. Install: https://doubleagent.so/install.md

Source: https://github.com/rom1504/img2dataset