What is Applebot-Extended? (Apple AI opt-out)

At a glance

Operator: Apple
Category: Training
Collects content used to train models. Blocking it keeps your work out of the training set, but does not affect whether the assistant can cite you live.
User-Agent token: None — this is a robots.txt control token, not a crawler.
Apple fetches with its own crawler and reads this token only to decide what it may then do with the result. No request ever carries the name Applebot-Extended, so it will never appear in your access logs no matter how often your content is used.
robots.txt token: Applebot-Extended
Respects robots.txt: Documented as compliant
Because Apple documents this as a robots.txt directive rather than as a separate crawler, it has no user-agent of its own and will not show up as a distinct hit in your logs.
Operator docs: support.apple.com

What Applebot-Extended does

Applebot-Extended does not crawl anything. Applebot does the fetching, for Siri, Spotlight and Safari suggestions; the extended token is the separate switch controlling whether that same content may also train Apple's generative models. Apple designed it as a directive rather than as a second crawler precisely so publishers would not have to choose between search and training.

Should you block Applebot-Extended?

Disallowing Applebot-Extended keeps your content out of Apple's model training while leaving Applebot free to index you for Siri and Spotlight. That makes it a cheap opt-out. Blocking Applebot itself is the much heavier move, and that is the one that removes you from Apple's search surfaces.

Our take

This is the low-risk opt-out. Because Applebot-Extended governs what may be done with content Apple has already fetched — rather than whether it may fetch at all — disallowing it keeps you in search while keeping you out of model training. That's the trade the token exists to let you make.

How to block Applebot-Extended

Add the rule below to the robots.txt file at the root of your domain (yourdomain.com/robots.txt). Use the Applebot-Extended token rather than the raw User-Agent string, and repeat the rule on every subdomain you want covered. New to the file? Read what robots.txt is first.

Block Applebot-Extended

robots.txt

User-agent: Applebot-Extended
Disallow: /

Allow Applebot-Extended explicitly

robots.txt

User-agent: Applebot-Extended
Allow: /

A note on what this does and doesn't do. robots.txt is a request, not a lock — it works because well-behaved crawlers choose to read it. It also isn't retroactive: anything already fetched stays fetched. If you need a hard stop, block at your firewall or CDN as well.

How MetaPilot handles Applebot-Extended

MetaPilot detects Applebot-Extended in your edge logs, so you see its real hits on your real pages rather than a guess: how often it visits, which URLs it takes, and how that trend moves week to week. From the same screen you flip Applebot-Extended between allow and deny with one click — MetaPilot rewrites only its own managed section of your robots.txt and serves it at the edge, so the change is live on the crawler's next visit with no deploy and no touching your own rules. See AI-crawler analytics or follow the guide to monitor and control AI crawlers.

Related crawlers

Applebot

Apple

Search engine

Apple's actual crawler — the one that does the fetching for Siri, Spotlight and Safari suggestions.

ClaudeBot

Anthropic

Training

Anthropic's bulk crawler, collecting public web content that may be used to develop and train Claude.

Bytespider

ByteDance

Training

ByteDance's web crawler, gathering content to train the large language models behind its products (ByteDance also owns TikTok).

CCBot (Common Crawl)

Common Crawl

Training

The crawler behind Common Crawl, the free public web archive that a great many AI models are trained on.

Browse every AI crawler

FAQ

What is Applebot-Extended?

Apple's AI opt-out token, telling Apple not to use content Applebot already crawled to train its foundation models.

Applebot-Extended does not crawl anything. Applebot does the fetching, for Siri, Spotlight and Safari suggestions; the extended token is the separate switch controlling whether that same content may also train Apple's generative models. Apple designed it as a directive rather than as a second crawler precisely so publishers would not have to choose between search and training.

Does Applebot-Extended respect robots.txt?

Yes — Apple documents that Applebot-Extended honours robots.txt. A Disallow for the Applebot-Extended token is respected.

Because Apple documents this as a robots.txt directive rather than as a separate crawler, it has no user-agent of its own and will not show up as a distinct hit in your logs.

How do I block Applebot-Extended?

Add these two lines to the robots.txt file at the root of your domain:

User-agent: Applebot-Extended
Disallow: /

Use the Applebot-Extended token, not the raw User-Agent string, and repeat the rule on every subdomain you want covered.

Will blocking Applebot-Extended hurt my traffic?

Disallowing Applebot-Extended keeps your content out of Apple's model training while leaving Applebot free to index you for Siri and Spotlight. That makes it a cheap opt-out. Blocking Applebot itself is the much heavier move, and that is the one that removes you from Apple's search surfaces.