Microsoft · Search indexes
msnbot
Legacy Microsoft search crawler still seen in logs. Allow it unless you have a specific reason to drop old Microsoft UAs.
- Operator
- Microsoft
- Traffic type
- Search indexes
- Verification
- User-agent + IP range
- robots.txt
- Honors robots.txt
Usually means: Crawlers that build the indexes ChatGPT search, Claude search, Perplexity, Google, and Bing draw from. Blocking them is how brands disappear from AI answers without noticing.
User-agent
robots.txt and most WAF rules match the token, not the full Mozilla string. Operators often wrap msnbot in extra product or version text.
msnbot
What msnbot does
msnbot is a legacy Microsoft indexer. Some Microsoft properties still emit it. Blocking it while allowing Bingbot is usually accidental.
How to get discovered
Prefer allowing Bingbot and msnbot together. Do not build robots groups that Disallow msnbot as a “bot cleanup” without checking Microsoft’s current crawler list.
Why msnbot might skip you
Aggressive “block old bots” rules, or copy-paste robots files from 2012 that Disallow msnbot while the rest of the site is open.
robots.txt rule
Allow this token on public pages you want retrieved or cited. A CDN “Block AI bots” toggle can still 403 it after robots.txt says Allow.
# Keep msnbot eligible to fetch this site
User-agent: msnbot
Allow: /Allowlist so msnbot can reach you
A user-agent is spoofable. Microsoft publishes current CIDR ranges as JSON — fetch that file for WAF/CDN allowlists instead of copying ranges from a blog. Ranges rotate; a screenshot of ten prefixes will go stale. If Cloudflare “Block AI bots” is on, this is the list that gets the real crawler through.
https://www.bing.com/toolbox/bingbot.json ↗
Sources
JavaScript: render behavior is not documented.
Other Microsoft bots
Training, search, and user-fetch tokens from the same operator are not interchangeable. Allow the discovery path even when you opt out of training.
msnbot FAQ
What is msnbot?
msnbot is a legacy Microsoft indexer. Some Microsoft properties still emit it. Blocking it while allowing Bingbot is usually accidental.
Should I allow msnbot in robots.txt?
Yes if you want Microsoft to be able to fetch and cite this site. Blocking msnbot is how pages stay invisible to that product even when Googlebot still crawls them.
How do I know a request is really msnbot?
A user-agent is a claim anyone can send. Match the msnbot token, then check the source IP against Microsoft’s published ranges. Use those ranges as a WAF allowlist so a “Block AI bots” rule does not 403 the real crawler.
Letting msnbot in is the start. Getting cited is the job.
Rankealo checks whether AI crawlers can reach you, then publishes pages built to be retrieved and quoted in ChatGPT, Claude, Perplexity, and Gemini.
