# See & control which AI bots crawl you

**Category:** Monitor · **Reading time:** 5 min

Answer engines can only cite what they read. This guide shows how to see which AI crawlers visit your site and decide, per bot, who gets in. New to AI crawlers? See *What is an LLM (and an answer engine)?* and *What is robots.txt?*

---

## Steps

### 1. Open your dashboard
See which AI bots have crawled your site — GPTBot (ChatGPT), ClaudeBot, PerplexityBot, Google-Extended and more — how often, and which of your pages they read most.

### 2. Read the training-vs-citation tradeoff
Blocking a bot keeps it out; allowing it can help you be cited in that engine's answers. MetaPilot flags what each bot is used for.

### 3. Allow or block in one click
Open **Agent Web → Robots & LLMs** and use the per-bot **Allow / Block** toggle. MetaPilot rewrites only its managed block in your robots.txt — your own rules are untouched.

### 4. It takes effect at the edge
The Cloudflare Worker serves the updated robots.txt on the bot's next crawl — no deploy required.

---

## See who is reading your site

Sign in and check your dashboard. [Get started →](https://metapilot.io)

---

> Stuck? Ask an AI to walk you through it: ChatGPT · Claude · Perplexity · Meta AI · Grok