What is meta-externalagent?
Crawls the web for uses such as training Meta's foundation AI models or improving products by indexing content directly.
- Operator
- Meta
- Purpose
- Mixed use
- Feeds
- Training of Meta foundation AI models, Meta products that index content directly
- robots.txt token
meta-externalagent- Follows robots.txt
- Yes
- IP ranges
- Not published
- Source
- Meta documentation
meta-externalagent user agent
meta-externalagent/1.1 (+/documentation/sharing/webmasters/web-crawlers)
Recorded exactly as published. The relative URL inside the string looks like an artifact of Meta's docs migration; don't rely on the URL portion and match on 'meta-externalagent' instead.
Match it in robots.txt with the token meta-externalagent. User agents can be faked, so check requests against the operator's verification method before trusting them. The current page publishes no IP list or reverse-DNS method; its allow-list link to 'IP addresses' leads only to the user-agent list. robots.txt changes can take up to 24 hours.
Should you block meta-externalagent?
Yes. Meta says meta-externalagent follows robots.txt. “In order to block these crawlers, add a disallow for the relevant crawler to robots.txt.”
Block meta-externalagent
User-agent: meta-externalagent Disallow: /
Allow meta-externalagent
User-agent: meta-externalagent Allow: /
meta-externalagent FAQ
What is meta-externalagent?
Crawls the web for uses such as training Meta's foundation AI models or improving products by indexing content directly.
What is the meta-externalagent user agent?
meta-externalagent identifies itself as: meta-externalagent/1.1 (+/documentation/sharing/webmasters/web-crawlers). In robots.txt, match it with the token "meta-externalagent".
Does meta-externalagent respect robots.txt?
Yes. Meta says meta-externalagent follows robots.txt.
How do I block meta-externalagent?
Add "User-agent: meta-externalagent" followed by "Disallow: /" to the robots.txt file at the root of your site.
Crawlable isn't the same as cited
Letting AI crawlers in is step one. Citedify checks whether ChatGPT, Claude, Perplexity and Google AI Overviews actually recommend your brand, and what to fix when they don't.
Facts on this page come from Meta's documentation, checked 2026-09-28. Sources: developers.facebook.com/documentation/sharing/webmasters/web-crawlers, developers.facebook.com/documentation/sharing/webmasters/web-crawlers.md