What are llms.txt and llms-full.txt?
TL;DR
llms.txt is a markdown file at your site's root that gives AI assistants a clean, curated map of your most important content. llms-full.txt is a longer companion with page-by-page detail. Together they help AI models understand and represent your site accurately.
What it is
Where robots.txt tells crawlers where they may go, and a sitemap lists every URL for search engines, llms.txt is different: it's a short, human-readable guidebook written for AI assistants. It highlights what your site is about and links to the pages that matter most, in plain markdown.
It's an emerging convention (see llmstxt.org), not yet a universal standard — but a small, low-cost investment as AI-driven traffic grows. llms-full.txt backs it up with fuller, per-page context for models that want more depth.
Why it matters
- Accuracy. A curated map reduces the chance an AI assistant misrepresents your content or cites the wrong page.
- Discoverability for AI. It points models straight at your best content instead of leaving them to guess from raw HTML.
- Freshness counts. A stale
llms.txtis worse than none — it sends models to content that moved or changed. Keeping it current is the whole game.
How MetaPilot handles it
MetaPilot pre-fills llms.txt and llms-full.txt from your sitemap data, lets you edit them, and serves them at the edge. Turn on auto-refresh and it rebuilds them whenever your content changes — so assistants never read a stale map. Set up llms.txt and llms-full.txt →
Related
FAQ
Is llms.txt the same as robots.txt?
No. robots.txt sets access rules for crawlers; llms.txt is a curated content map written for AI assistants.
Is llms.txt an official standard?
It's an emerging convention (llmstxt.org), increasingly recognized but not universally adopted — cheap to provide now.
What's the difference between llms.txt and llms-full.txt?
llms.txt is the concise map; llms-full.txt adds richer page-by-page detail.
Do I have to keep it updated?
Yes — a stale map misleads models. MetaPilot can auto-refresh it for you.
Ask an AI to explain this