Identity: Signs its requests (Web Bot Auth).
How to recognise it
- Signed requests:
Signature-Agentfromagent.bot.goog, keys athttps://agent.bot.goog/.well-known/http-message-signatures-directory - Published IP list: https://developers.google.com/static/crawling/ipranges/user-triggered-agents.json
- Reverse DNS ends in
googlebot.com,google.com,googleusercontent.com(then forward-confirm) - User agent contains
Google-Agent
With Double Agent
Sessions from Google-Agent are labelled agent with agent_id: google.google-agent, and tagged in your analytics. Nothing is blocked unless you choose to. Install: https://doubleagent.so/install.md
Source: https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers ยท Operator docs: https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers