What is Google-Extended?
A robots.txt control token with no crawler of its own: it tells Google whether content it crawls may be used to train future Gemini models and to ground Gemini answers.
- Operator
- Purpose
- Mixed use
- Feeds
- Training of future Gemini models (Gemini Apps, Vertex AI API for Gemini), Grounding in Gemini Apps, Grounding with Google Search on Vertex AI
- robots.txt token
Google-Extended- Follows robots.txt
- Yes
- IP ranges
- Not published
- Source
- Google documentation
Google-Extended user agent
Google doesn't publish a full user-agent string for Google-Extended.
Match it in robots.txt with the token Google-Extended. User agents can be faked, so check requests against the operator's verification method before trusting them. Not applicable: Google-Extended never appears as its own user agent. Requests come from other Google crawlers, which can be verified via common-crawlers.json and googlebot.com reverse DNS.
Should you block Google-Extended?
Disallowing Google-Extended keeps content out of Gemini model training and Gemini/Vertex AI grounding; it does not affect inclusion in Google Search and is not a Search ranking signal.
Yes. Google says Google-Extended follows robots.txt. Google's common crawlers “always obey robots.txt rules when crawling automatically”; this token “is used in a control capacity.”
Block Google-Extended
User-agent: Google-Extended Disallow: /
Allow Google-Extended
User-agent: Google-Extended Allow: /
Other Google crawlers
Google uses separate user agents for separate jobs, so you can allow one and block another.
- GoogleOther (Mixed use): Google's generic crawler that various product teams use to fetch public content, for example one-off crawls for internal research and development.
Google-Extended FAQ
What is Google-Extended?
A robots.txt control token with no crawler of its own: it tells Google whether content it crawls may be used to train future Gemini models and to ground Gemini answers.
What is the Google-Extended user agent?
Google doesn't publish a full user-agent string. In robots.txt, use the token "Google-Extended".
Does Google-Extended respect robots.txt?
Yes. Google says Google-Extended follows robots.txt.
How do I block Google-Extended?
Add "User-agent: Google-Extended" followed by "Disallow: /" to the robots.txt file at the root of your site. Disallowing Google-Extended keeps content out of Gemini model training and Gemini/Vertex AI grounding; it does not affect inclusion in Google Search and is not a Search ranking signal.
Crawlable isn't the same as cited
Letting AI crawlers in is step one. Citedify checks whether ChatGPT, Claude, Perplexity and Google AI Overviews actually recommend your brand, and what to fix when they don't.
Facts on this page come from Google's documentation, checked 2026-09-28. Sources: developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers