Meta · AI answers
meta-externalfetcher
Meta’s fetcher when a person requests or shares a specific URL. Blocking it breaks Meta AI reads and some share previews.
- Operator
- Meta
- Traffic type
- AI answers
- Verification
- Not independently verifiable
- robots.txt
- Usually honors robots.txt
Usually means: Live fetches when someone asks an assistant a question. If these bots cannot reach the page, you cannot be the citation in that answer.
User-agent
robots.txt and most WAF rules match the token, not the full Mozilla string. Operators often wrap meta-externalfetcher in extra product or version text.
meta-externalfetcher
What meta-externalfetcher does
meta-externalfetcher retrieves a URL because a Meta product needs that document now — a user request, a share, or an assistant lookup.
How to get discovered
Allow meta-externalfetcher, keep Open Graph tags honest, and do not require a Facebook login to read the public article body.
Why meta-externalfetcher might skip you
Aggressive bot management against Meta UAs, or robots groups that Disallow meta-* after a training opt-out that was copied too broadly.
robots.txt rule
Allow this token on public pages you want retrieved or cited. A CDN “Block AI bots” toggle can still 403 it after robots.txt says Allow.
# Keep meta-externalfetcher eligible to fetch this site
User-agent: meta-externalfetcher
Allow: /Meta has not published a machine-readable range file for this token. Do not treat the user-agent alone as proof of identity.
JavaScript: render behavior is not documented.
Other Meta bots
Training, search, and user-fetch tokens from the same operator are not interchangeable. Allow the discovery path even when you opt out of training.
meta-externalfetcher FAQ
What is meta-externalfetcher?
meta-externalfetcher retrieves a URL because a Meta product needs that document now — a user request, a share, or an assistant lookup.
Should I allow meta-externalfetcher in robots.txt?
Yes if you want Meta to be able to fetch and cite this site. Blocking meta-externalfetcher is how pages stay invisible to that product even when Googlebot still crawls them.
How do I know a request is really meta-externalfetcher?
Meta has not published a range file for this token. Treat the user-agent as a claimed identity, and do not build a hard IP allowlist from blog posts or screenshots.
Letting meta-externalfetcher in is the start. Getting cited is the job.
Rankealo checks whether AI crawlers can reach you, then publishes pages built to be retrieved and quoted in ChatGPT, Claude, Perplexity, and Gemini.
