# Applebot **Search engine** · Operated by Apple Apple's actual crawler — the one that does the fetching for Siri, Spotlight and Safari suggestions. --- ## At a glance | Property | Value | |---|---| | Operator | Apple | | Category | Search engine | | User-Agent token | `Applebot` | | robots.txt token | `Applebot` | | Respects robots.txt | Documented as compliant | | Operator docs | support.apple.com | > **Note:** A classic search crawler whose index also feeds that company's AI answers. Blocking it removes you from both the blue links and the AI summaries above them. > > Applebot-Extended shows as "policy only" because it has no user agent; Applebot is the one that will actually appear in your logs. --- ## What Applebot does Applebot indexes the web for Apple's own search surfaces: Siri answers, Spotlight results and the suggestions Safari shows as you type. The corpus it builds is also the corpus Apple's foundation models may train on, unless you disallow that separately with Applebot-Extended. --- ## Should you block Applebot? Blocking Applebot removes you from Siri and Spotlight entirely. If your goal is to stay out of Apple's model training, block Applebot-Extended instead — that is precisely the choice Apple split into two tokens so you would not have to give up search to opt out of training. **Our take:** Don't. Blocking Applebot removes you from Apple altogether — the ordinary results and the AI answers built on top of them. If what you actually want is to stay in search while staying out of model training, that is a separate token (Applebot-Extended), and it's the one to reach for. MetaPilot shows Applebot in your crawler analytics but won't write a block for it from the dashboard. --- ## The robots.txt rules for Applebot Add the rule below to the `robots.txt` file at the root of your domain (`yourdomain.com/robots.txt`). Use the `Applebot` token rather than the raw User-Agent string, and repeat the rule on every subdomain you want covered. > ⚠️ **The allow rule is the one you want.** The disallow variant is documented below for completeness, but it removes your site from Apple entirely. Don't paste it unless that is genuinely what you intend. **Allow Applebot explicitly:** ``` User-agent: Applebot Allow: / ``` **Block Applebot — removes you from Apple:** ``` User-agent: Applebot Disallow: / ``` **A note on what this does and doesn't do.** `robots.txt` is a request, not a lock — it works because well-behaved crawlers choose to read it. It also isn't retroactive: anything already fetched stays fetched. If you need a hard stop, block at your firewall or CDN as well. --- ## How MetaPilot handles Applebot MetaPilot detects Applebot in your edge logs, so you see its real hits on your real pages rather than a guess: how often it visits, which URLs it takes, and how that trend moves week to week. From the same screen you flip Applebot between allow and deny with one click — MetaPilot rewrites only its own managed section of your `robots.txt` and serves it at the edge, so the change is live on the crawler's next visit with no deploy and no touching your own rules. --- ## Related crawlers - **Applebot-Extended** (Apple) — Training: Apple's AI opt-out token, telling Apple not to use content Applebot already crawled to train its foundation models. - **Google-InspectionTool** (Google) — Search engine: The fetcher behind your own Search Console testing tools — URL Inspection and the Rich Results Test. - **Googlebot** (Google) — Search engine: Google's main search crawler — and, indirectly, the crawler behind AI Overviews, because those are generated from the index it builds. - **Googlebot-Image** (Google) — Search engine: Googlebot's image-fetching counterpart, which populates Google Images. --- ## FAQ **What is Applebot?** Apple's actual crawler — the one that does the fetching for Siri, Spotlight and Safari suggestions. Applebot indexes the web for Apple's own search surfaces: Siri answers, Spotlight results and the suggestions Safari shows as you type. The corpus it builds is also the corpus Apple's foundation models may train on, unless you disallow that separately with Applebot-Extended. **Does Applebot respect robots.txt?** Yes — Apple documents that Applebot honours `robots.txt`. A `Disallow` for the `Applebot` token is respected. Applebot-Extended shows as "policy only" because it has no user agent; Applebot is the one that will actually appear in your logs. **Should I block Applebot?** Almost certainly not. Disallowing `Applebot` removes you from Apple's search results, and because those same results feed Apple's AI answers, you lose both at once. If you want to stay in search but out of generative AI, that is a different token — see the training opt-out for Apple (Applebot-Extended). MetaPilot will not write a block for Applebot from the dashboard for this reason. **Will blocking Applebot hurt my traffic?** Blocking Applebot removes you from Siri and Spotlight entirely. If your goal is to stay out of Apple's model training, block Applebot-Extended instead — that is precisely the choice Apple split into two tokens so you would not have to give up search to opt out of training.