llms.txt: What It Is, What It Is Not, and Whether You Should Ship One

llms.txt is a proposed convention: a plain-text markdown file at the root of your domain that gives language models a curated summary of your site. A short description of who you are, followed by an annotated list of your most important pages. Think of it as a sitemap written for a reader that understands prose, not XML.

The idea behind it

Context windows are finite and crawling is expensive. When an AI system wants to understand a site, it either reads pages one by one (slow, and it may pick badly) or it reads a file the site owner wrote specifically to orient it. llms.txt is the second option: you choose what a model sees first, and you phrase your own description instead of hoping the model assembles a good one from scattered pages.

The format is deliberately trivial. An H1 with the site name, a blockquote summary, then sections of links with one-line annotations. Some sites add an llms-full.txt containing expanded content for systems willing to read more.

What it does not do

Honesty matters here, because the discourse has run ahead of reality:

• It is not a ranking signal. No engine has announced that llms.txt affects retrieval or citation.
• It is not universally read. Adoption among AI companies is uneven and mostly undocumented. Some crawlers fetch it; there is no guarantee any given engine uses what it fetches.
• It is not a substitute for crawlable pages. If your actual content is unreadable to bots, a tidy manifest will not save you.

Why we ship it anyway

The cost is roughly an hour, and the asymmetry is attractive. Three practical arguments:

• You control the first impression. If any system does read it, it gets your description of your business in your words: exactly the liftable phrasing you want models to repeat.
• It concentrates your best pages. Agents and retrieval systems that do honor it skip the crawl-and-guess phase and land directly on the pages you chose.
• It future-proofs cheaply. Standards like robots.txt and sitemaps also started as informal conventions that some crawlers honored and others ignored. The sites that adopted early lost nothing.

Doing it properly

Write the summary the way you would brief a new employee: what you do, for whom, and what makes the claim credible. Annotate each link with why it matters, not just its title. Keep it current: a stale llms.txt that contradicts your site is worse than none, because inconsistency is exactly what erodes machine trust.

And keep perspective. This file is a supporting detail in an AEO program, not a strategy. The load-bearing work is still retrievability, liftable facts, and corroboration across the web. Ship llms.txt in the same spirit you ship a favicon: small, correct, and done, then move back to the work that moves answers, like schema markup that machines actually use.