# ChatGPT-User **Answer engine · Operated by OpenAI** The fetcher ChatGPT uses to open a specific page in real time while answering someone. --- ## At a glance | Property | Value | |---|---| | **Operator** | OpenAI | | **Category** | Answer engine | | **User-Agent token** | `ChatGPT-User` | | **robots.txt token** | `ChatGPT-User` | | **Respects robots.txt** | Not reliably | | **Operator docs** | developers.openai.com | **Category note:** Fetches your page in real time to answer a user who is asking right now. Blocking it removes you from those answers — this is the one to leave open. **robots.txt note:** OpenAI's own documentation says that because these fetches are initiated by a user rather than by an automatic crawl, "robots.txt rules may not apply" to ChatGPT-User. A Disallow for this token is best read as a request OpenAI may honour, not a hard block. --- ## What ChatGPT-User does When a ChatGPT user — or a custom GPT, or an app calling OpenAI's browsing tools — needs the contents of a particular URL, ChatGPT-User fetches that one page on the spot. It is not a bulk crawler and it is not what decides whether you appear in ChatGPT search. The page it retrieves can end up quoted and linked in the answer the person sees. --- ## Should you block ChatGPT-User? Blocking ChatGPT-User is the expensive one. It removes your page from an answer being written for a real person who is asking about your subject right now, and it takes the citation link with it. Very few sites come out ahead on that trade. > **Our take:** Leave it open. Answer-engine fetchers only run because a person just asked a question, and the page they read is the page that gets cited. Being the source is the entire goal — blocking here buys you nothing and costs you the mention. --- ## How to block ChatGPT-User Add the rule below to the `robots.txt` file at the root of your domain (`yourdomain.com/robots.txt`). Use the `ChatGPT-User` token rather than the raw User-Agent string, and repeat the rule on every subdomain you want covered. **Block ChatGPT-User:** ``` User-agent: ChatGPT-User Disallow: / ``` **Allow ChatGPT-User explicitly:** ``` User-agent: ChatGPT-User Allow: / ``` > **Note:** `robots.txt` is a request, not a lock — it works because well-behaved crawlers choose to read it. It also isn't retroactive: anything already fetched stays fetched. If you need a hard stop, block at your firewall or CDN as well. --- ## How MetaPilot handles ChatGPT-User MetaPilot detects ChatGPT-User in your edge logs, so you see its real hits on your real pages rather than a guess: how often it visits, which URLs it takes, and how that trend moves week to week. From the same screen you flip ChatGPT-User between allow and deny with one click — MetaPilot rewrites only its own managed section of your `robots.txt` and serves it at the edge, so the change is live on the crawler's next visit with no deploy and no touching your own rules. --- ## Related crawlers - **GPTBot** (OpenAI) — Training: OpenAI's bulk web crawler, collecting public pages that may be used to train future OpenAI models. - **OAI-SearchBot** (OpenAI) — Search index: The crawler that builds the search index behind ChatGPT's search results. - **Claude-User** (Anthropic) — Answer engine: The fetcher Claude uses to read a specific page while answering a user who asked about it. - **DuckAssistBot** (DuckDuckGo) — Answer engine: DuckDuckGo's real-time fetcher for DuckAssist, the AI-assisted answers that appear above its search results. --- ## FAQ **What is ChatGPT-User?** The fetcher ChatGPT uses to open a specific page in real time while answering someone. When a ChatGPT user — or a custom GPT, or an app calling OpenAI's browsing tools — needs the contents of a particular URL, ChatGPT-User fetches that one page on the spot. It is not a bulk crawler and it is not what decides whether you appear in ChatGPT search. The page it retrieves can end up quoted and linked in the answer the person sees. **Does ChatGPT-User respect robots.txt?** Not reliably. A `Disallow` for `ChatGPT-User` is worth adding, but it should not be treated as a guarantee. OpenAI's own documentation says that because these fetches are initiated by a user rather than by an automatic crawl, "robots.txt rules may not apply" to ChatGPT-User. A Disallow for this token is best read as a request OpenAI may honour, not a hard block. **How do I block ChatGPT-User?** Add these two lines to the `robots.txt` file at the root of your domain: `User-agent: ChatGPT-User` `Disallow: /` Use the `ChatGPT-User` token, not the raw User-Agent string, and repeat the rule on every subdomain you want covered. **Will blocking ChatGPT-User hurt my traffic?** Blocking ChatGPT-User is the expensive one. It removes your page from an answer being written for a real person who is asking about your subject right now, and it takes the citation link with it. Very few sites come out ahead on that trade.