xAI · Other AI bots
xAI-Grok
Another xAI/Grok identity in logs. Rankealo lists it so you can allow the real token instead of guessing.
- Operator
- xAI
- Traffic type
- Other AI bots
- Verification
- Not independently verifiable
- robots.txt
- Compliance not documented
Usually means: Link-preview, ads, and product fetchers. Allow them if you want unfurls and landing-page fetches to work; they are not the citation path.
User-agent
robots.txt and most WAF rules match the token, not the full Mozilla string. Operators often wrap xAI-Grok in extra product or version text.
xAI-Grok
What xAI-Grok does
xAI-Grok is a documented-in-the-wild UA for Grok-related fetches. Matching robots groups is token-exact — xAI-SearchBot will not match xAI-Grok.
How to get discovered
Name xAI-Grok in robots.txt if you see it in logs. Serve HTML; Grok will not run your SPA to invent a citation.
Why xAI-Grok might skip you
You allowed xAI-SearchBot only, and this token still 403s at the edge.
robots.txt rule
Allow this token on public pages you want retrieved or cited. A CDN “Block AI bots” toggle can still 403 it after robots.txt says Allow.
# Keep xAI-Grok eligible to fetch this site
User-agent: xAI-Grok
Allow: /xAI has not published a machine-readable range file for this token. Do not treat the user-agent alone as proof of identity.
JavaScript: reads raw HTML only.
Other xAI bots
Training, search, and user-fetch tokens from the same operator are not interchangeable. Allow the discovery path even when you opt out of training.
xAI-Grok FAQ
What is xAI-Grok?
xAI-Grok is a documented-in-the-wild UA for Grok-related fetches. Matching robots groups is token-exact — xAI-SearchBot will not match xAI-Grok.
Should I allow xAI-Grok in robots.txt?
Yes if you want xAI to be able to fetch and cite this site. Blocking xAI-Grok is how pages stay invisible to that product even when Googlebot still crawls them.
How do I know a request is really xAI-Grok?
xAI has not published a range file for this token. Treat the user-agent as a claimed identity, and do not build a hard IP allowlist from blog posts or screenshots.
Letting xAI-Grok in is the start. Getting cited is the job.
Rankealo checks whether AI crawlers can reach you, then publishes pages built to be retrieved and quoted in ChatGPT, Claude, Perplexity, and Gemini.
