# Perplexity-User

**Answer engine** · Operated by Perplexity

Perplexity's live fetcher, which opens a page in the moment a user's question requires it.

---

## At a glance

| Property | Value |
|---|---|
| **Operator** | Perplexity |
| **Category** | Answer engine |
| **User-Agent token** | `Perplexity-User` |
| **robots.txt token** | `Perplexity-User` |
| **Respects robots.txt** | Not reliably |
| **Operator docs** | docs.perplexity.ai |

**Category note:** Fetches your page in real time to answer a user who is asking right now. Blocking it removes you from those answers — this is the one to leave open.

**robots.txt note:** Perplexity's own documentation states that because a user requested the fetch, "this fetcher generally ignores robots.txt rules". That is a deliberate design position, not an accident — and it is the same position Cloudflare disputed publicly in August 2025.

---

## What Perplexity-User does

Perplexity-User retrieves a specific URL while a user is waiting on an answer, rather than crawling on a schedule. Perplexity treats it as an extension of the user rather than as a crawler, which is also how it justifies its robots.txt behaviour. Pages it fetches can be quoted and linked in the response.

---

## Should you block Perplexity-User?

Blocking it means Perplexity stops opening your pages for the person who asked — though because of the robots.txt position, a Disallow may not actually stop it. If you want a reliable block you need a firewall or CDN rule, and you should be sure you want to be absent from those answers.

> **Our take:** Leave it open. Answer-engine fetchers only run because a person just asked a question, and the page they read is the page that gets cited. Being the source is the entire goal — blocking here buys you nothing and costs you the mention.

---

## How to block Perplexity-User

Add the rule below to the `robots.txt` file at the root of your domain (`yourdomain.com/robots.txt`). Use the `Perplexity-User` token rather than the raw User-Agent string, and repeat the rule on every subdomain you want covered.

**Block Perplexity-User:**

```
User-agent: Perplexity-User
Disallow: /
```

**Allow Perplexity-User explicitly:**

```
User-agent: Perplexity-User
Allow: /
```

**Note:** `robots.txt` is a request, not a lock — it works because well-behaved crawlers choose to read it. It also isn't retroactive: anything already fetched stays fetched. If you need a hard stop, block at your firewall or CDN as well.

---

## How MetaPilot handles Perplexity-User

MetaPilot detects Perplexity-User in your edge logs, so you see its real hits on your real pages rather than a guess: how often it visits, which URLs it takes, and how that trend moves week to week. From the same screen you flip Perplexity-User between allow and deny with one click — MetaPilot rewrites only its own managed section of your `robots.txt` and serves it at the edge, so the change is live on the crawler's next visit with no deploy and no touching your own rules.

---

## Related crawlers

- **PerplexityBot** (Perplexity) — Search index. Perplexity's indexing crawler — the one that decides whether your pages can be surfaced as a source in Perplexity answers.
- **Claude-User** (Anthropic) — Answer engine. The fetcher Claude uses to read a specific page while answering a user who asked about it.
- **DuckAssistBot** (DuckDuckGo) — Answer engine. DuckDuckGo's real-time fetcher for DuckAssist, the AI-assisted answers that appear above its search results.
- **Gemini-Deep-Research** (Google) — Answer engine. The agent behind Gemini's Deep Research mode, which reads pages as it works through a multi-step research task for one user.

---

## FAQ

**What is Perplexity-User?**
Perplexity's live fetcher, which opens a page in the moment a user's question requires it. Perplexity-User retrieves a specific URL while a user is waiting on an answer, rather than crawling on a schedule. Perplexity treats it as an extension of the user rather than as a crawler, which is also how it justifies its robots.txt behaviour. Pages it fetches can be quoted and linked in the response.

**Does Perplexity-User respect robots.txt?**
Not reliably. A `Disallow` for `Perplexity-User` is worth adding, but it should not be treated as a guarantee. Perplexity's own documentation states that because a user requested the fetch, "this fetcher generally ignores robots.txt rules". That is a deliberate design position, not an accident — and it is the same position Cloudflare disputed publicly in August 2025.

**How do I block Perplexity-User?**
Add these two lines to the `robots.txt` file at the root of your domain:
`User-agent: Perplexity-User`
`Disallow: /`
Use the `Perplexity-User` token, not the raw User-Agent string, and repeat the rule on every subdomain you want covered.

**Will blocking Perplexity-User hurt my traffic?**
Blocking it means Perplexity stops opening your pages for the person who asked — though because of the robots.txt position, a Disallow may not actually stop it. If you want a reliable block you need a firewall or CDN rule, and you should be sure you want to be absent from those answers.