Directory Crawlers

Baiduspider Bot

Builds a search index. Operated by baidu.

Identity: Verifiable by reverse DNS.

How to recognise it

  • Reverse DNS ends in baidu.com, baidu.jp (then forward-confirm)

With Double Agent

Sessions from Baiduspider are labelled bot with agent_id: baidu.baiduspider, and tagged in your analytics. Nothing is blocked unless you choose to. Install: https://doubleagent.so/install.md

Source: https://help.baidu.com/question?prod_id=99&class=476&id=2996