Google · AI answers
Google-Agent
A Google user-triggered fetcher used by AI and product experiences. If Google-Agent cannot read you, those surfaces have nothing to quote.
- Operator
- Traffic type
- AI answers
- Verification
- User-agent + IP range
- robots.txt
- Usually honors robots.txt
Usually means: Live fetches when someone asks an assistant a question. If these bots cannot reach the page, you cannot be the citation in that answer.
User-agent
robots.txt and most WAF rules match the token, not the full Mozilla string. Operators often wrap Google-Agent in extra product or version text.
Google-Agent
What Google-Agent does
Google-Agent fetches a page because a Google AI or assistant experience needs the live document, not because Search is recrawling the site. Treat it as an answer-path UA, distinct from Googlebot.
How to get discovered
Do not blanket-block Google AI user-agents at the WAF. Keep the canonical HTML complete, with Organization/WebPage schema so the agent can resolve who you are.
Why Google-Agent might skip you
“Block AI bots” toggles, bot-management scores that treat Google-Agent as a scraper, or a robots group that Disallows unknown Google UAs while still allowing Googlebot.
robots.txt rule
Allow this token on public pages you want retrieved or cited. A CDN “Block AI bots” toggle can still 403 it after robots.txt says Allow.
# Keep Google-Agent eligible to fetch this site
User-agent: Google-Agent
Allow: /Allowlist so Google-Agent can reach you
A user-agent is spoofable. Google publishes current CIDR ranges as JSON — fetch that file for WAF/CDN allowlists instead of copying ranges from a blog. Ranges rotate; a screenshot of ten prefixes will go stale. If Cloudflare “Block AI bots” is on, this is the list that gets the real crawler through.
https://developers.google.com/static/search/apis/ipranges/user-triggered-fetchers.json ↗
Sources
JavaScript: render behavior is not documented.
Other Google bots
Training, search, and user-fetch tokens from the same operator are not interchangeable. Allow the discovery path even when you opt out of training.
Google-Agent FAQ
What is Google-Agent?
Google-Agent fetches a page because a Google AI or assistant experience needs the live document, not because Search is recrawling the site. Treat it as an answer-path UA, distinct from Googlebot.
Should I allow Google-Agent in robots.txt?
Yes if you want Google to be able to fetch and cite this site. Blocking Google-Agent is how pages stay invisible to that product even when Googlebot still crawls them.
How do I know a request is really Google-Agent?
A user-agent is a claim anyone can send. Match the Google-Agent token, then check the source IP against Google’s published ranges. Use those ranges as a WAF allowlist so a “Block AI bots” rule does not 403 the real crawler.
Letting Google-Agent in is the start. Getting cited is the job.
Rankealo checks whether AI crawlers can reach you, then publishes pages built to be retrieved and quoted in ChatGPT, Claude, Perplexity, and Gemini.
