You.com · Search indexes
YouBot
You.com’s search/index crawler. Allow it if you want You.com AI search to be able to retrieve your pages.
- Operator
- You.com
- Traffic type
- Search indexes
- Verification
- Not independently verifiable
- robots.txt
- Usually honors robots.txt
Usually means: Crawlers that build the indexes ChatGPT search, Claude search, Perplexity, Google, and Bing draw from. Blocking them is how brands disappear from AI answers without noticing.
User-agent
robots.txt and most WAF rules match the token, not the full Mozilla string. Operators often wrap YouBot in extra product or version text.
YouBot
What YouBot does
YouBot crawls to keep You.com’s search and answer index current. It is a discovery bot for that engine, not a Google ranking signal.
How to get discovered
Allow YouBot, serve HTML, and keep canonical URLs stable. You.com cannot cite a URL it is not allowed to fetch.
Why YouBot might skip you
Unknown-bot WAF defaults, or robots files that only list the four “big” AI companies.
robots.txt rule
Allow this token on public pages you want retrieved or cited. A CDN “Block AI bots” toggle can still 403 it after robots.txt says Allow.
# Keep YouBot eligible to fetch this site
User-agent: YouBot
Allow: /You.com has not published a machine-readable range file for this token. Do not treat the user-agent alone as proof of identity.
JavaScript: reads raw HTML only.
YouBot FAQ
What is YouBot?
YouBot crawls to keep You.com’s search and answer index current. It is a discovery bot for that engine, not a Google ranking signal.
Should I allow YouBot in robots.txt?
Yes if you want You.com to be able to fetch and cite this site. Blocking YouBot is how pages stay invisible to that product even when Googlebot still crawls them.
How do I know a request is really YouBot?
You.com has not published a range file for this token. Treat the user-agent as a claimed identity, and do not build a hard IP allowlist from blog posts or screenshots.
Letting YouBot in is the start. Getting cited is the job.
Rankealo checks whether AI crawlers can reach you, then publishes pages built to be retrieved and quoted in ChatGPT, Claude, Perplexity, and Gemini.
