Perplexity · Search indexes
PerplexityBot
Perplexity’s index crawler. This is how pages enter the corpus Perplexity answers from — a search bot, not a training bot.
- Operator
- Perplexity
- Traffic type
- Search indexes
- Verification
- Not independently verifiable
- robots.txt
- Honors robots.txt
Usually means: Crawlers that build the indexes ChatGPT search, Claude search, Perplexity, Google, and Bing draw from. Blocking them is how brands disappear from AI answers without noticing.
User-agent
robots.txt and most WAF rules match the token, not the full Mozilla string. Operators often wrap PerplexityBot in extra product or version text.
PerplexityBot
Example full string: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot
What PerplexityBot does
PerplexityBot keeps Perplexity’s answer and search index fresh. Citations in Perplexity usually require this bot (or Perplexity-User) to have actually read the URL.
How to get discovered
Allow PerplexityBot, ship HTML answers, and internally link the pages you want retrieved. Perplexity rewards direct, quotable pages over thin tag archives.
Why PerplexityBot might skip you
robots Disallow, CDN AI-bot blocks, or client-rendered content. A site that ranks in Google can still look empty to PerplexityBot.
robots.txt rule
Allow this token on public pages you want retrieved or cited. A CDN “Block AI bots” toggle can still 403 it after robots.txt says Allow.
# Keep PerplexityBot eligible to fetch this site
User-agent: PerplexityBot
Allow: /Perplexity has not published a machine-readable range file for this token. Do not treat the user-agent alone as proof of identity.
Sources
JavaScript: reads raw HTML only.
Other Perplexity bots
Training, search, and user-fetch tokens from the same operator are not interchangeable. Allow the discovery path even when you opt out of training.
PerplexityBot FAQ
What is PerplexityBot?
PerplexityBot keeps Perplexity’s answer and search index fresh. Citations in Perplexity usually require this bot (or Perplexity-User) to have actually read the URL.
Should I allow PerplexityBot in robots.txt?
Yes if you want Perplexity to be able to fetch and cite this site. Blocking PerplexityBot is how pages stay invisible to that product even when Googlebot still crawls them.
How do I know a request is really PerplexityBot?
Perplexity has not published a range file for this token. Treat the user-agent as a claimed identity, and do not build a hard IP allowlist from blog posts or screenshots.
Letting PerplexityBot in is the start. Getting cited is the job.
Rankealo checks whether AI crawlers can reach you, then publishes pages built to be retrieved and quoted in ChatGPT, Claude, Perplexity, and Gemini.
