Add .md to any URL on your site. Get the page, without the page.
LLMs read your HTML the hard way: through nav bars, cookie banners, and 400KB of framework. MetaPilot serves a clean Markdown version of any page to anything that explicitly asks for it — by URL suffix or by Accept header. Same content, none of the noise.
200 OK content-type: text/markdown x-markdown-tokens: 412 (html: 38,900) # Pricing Plans for teams of every size. Billed monthly or annually. ## Growth — $99/mo - 500 pages - 500 regenerations / month - 90-day crawler history # no nav · no banner · no footer · no scripts
The problem
An AI agent fetching your page gets everything: the skip links, the mega-menu, the newsletter modal, the footer sitemap, the analytics scripts. The actual content is often under 5% of the bytes — and models with a limited context window simply truncate the rest.
The obvious fix is to detect bots and serve them something cleaner. That is cloaking, and it is the kind of thing that gets sites penalised. So you need a way to serve Markdown to AI that anyone can request, not one that only fires for user agents you recognise.
How it works
-
The agent asks explicitly
Either
https://yoursite.com/pricing.md, or an ordinary request to/pricingcarryingAccept: text/markdown. Two ways to ask, both open to anyone. -
The Worker recognises the signal
Your edge Worker spots the suffix or the header and asks MetaPilot for that page’s Markdown — before the request ever reaches your origin.
-
Clean Markdown comes back
Headings, prose, lists, specs, price and availability. No nav, no cookie banner, no footer sitemap. If we have no Markdown for that URL, the request falls through to your normal page.
What you get
Not cloaking, by construction
We respond to an explicit request signal, never to a user-agent string. You, a browser extension, a script, an agent — everyone who asks the same way gets the same answer. The honest version of this needs something in the request path that can branch on a header. That is the Worker.
LLM-friendly content, not a 400KB page
Every token an agent spends parsing your mega-menu is a token it does not spend on your product. Markdown for LLMs cuts the page down to the part that actually says something.
Generated with your metadata, not separately
Markdown comes out of the same pipeline and the same page fetch as your structured data. It cannot describe a different version of the page, because it never saw a different version.
Editable per page
Do not like what was generated for a URL? Overwrite it in the portal and we serve yours verbatim, exactly as written.
Falls back safely
No Markdown for a URL means nothing changes: the .md address redirects to the normal page, and an Accept-negotiated request just gets your HTML. Responses carry Vary: Accept so caches keep the two representations apart.
Nothing to build
No second content pipeline, no static export, no /md/ directory to keep in sync with the real site. It is served, not generated into your repo.
Plan availability
Markdown comes out of the standard metadata pipeline, on the same fetch and the same AI call as your JSON-LD and Open Graph, so it never costs a generation of its own. It follows the one number your plan is priced on: pages under management, from 5 free to 2,000 on Business.
Every plan generates it. Where it is served depends on your channel: from your own /page.md URLs on Cloudflare, and on Shopify and Wix from a MetaPilot URL that each page links to with rel="alternate".
Frequently asked
It would be, if we sniffed user agents. We do not. We respond to an explicit .md suffix or Accept: text/markdown, and both are available to every client equally — including you, in a terminal, right now.
No. A normal browser or crawler request gets your normal HTML, unchanged.
Yes, per page. An edited version is served exactly as you wrote it, and later regenerations do not silently discard your edit without you seeing it.
Not yet, and we would rather say so. The HTML is stripped of attributes before the model reads it, so generated Markdown carries headings, prose, lists, specs and breadcrumbs but no inline hyperlinks. If a page needs links, edit its Markdown by hand and we serve yours verbatim.
Partly, and the difference is where the file lives. Serving /pricing.md from your own domain means branching on the request before it reaches your origin, which needs a Cloudflare zone. On Shopify and Wix we generate the same Markdown and advertise it from each page with a rel="alternate" link to a MetaPilot URL, so an agent still finds it. Same content, different address. The delivery table on pricing is the authoritative version of this answer.
The .md URL redirects to the normal page — a temporary redirect, so a page that gains Markdown later is not permanently cached away from it. An Accept: text/markdown request simply gets your normal HTML.
llms.txt is a site-level map telling an agent what exists. This is the page-level content it reads once it gets there. They work together.
Keep reading
Give agents the content, not the chrome
Markdown is generated alongside your structured data, so turning it on is not a second project.
Ask an AI why Markdown beats HTML for agents