Directory Crawlers

AzureAI-SearchBot Bot

Builds a search index. Operated by microsoft.

Identity: Names itself in its user agent only.

How to recognise it

  • User agent contains AzureAI-SearchBot

With Double Agent

Sessions from AzureAI-SearchBot are labelled bot with agent_id: microsoft.azureai-searchbot, and tagged in your analytics. Nothing is blocked unless you choose to. Install: https://doubleagent.so/install.md

Source: https://github.com/ai-robots-txt/ai.robots.txt