llms.txt is a plain Markdown file you put at the root of your site, at /llms.txt, that gives AI tools a curated map of your most important pages in a format they can read cheaply. Think of it as a table of contents written for large language models rather than for search crawlers. It was proposed by Jeremy Howard of Answer.AI in late 2024, and through 2025 it became a quiet standard among documentation and developer-tool sites. Here is what it is, how to write one, and the honest answer on whether it actually does anything yet.
What is llms.txt?
llms.txt is a Markdown file at your site root that lists your key pages with short descriptions, so an AI model can find and understand your content without crawling and parsing your full HTML. The format is deliberately simple: an H1 with the site or project name, a blockquote summary, then headed sections of links. Because it is Markdown, a model can read it in a fraction of the tokens it would spend rendering a page, which is the whole point. You can see this site's own version at /llms.txt.
llms.txt vs llms-full.txt
The difference is depth. llms.txt is the map: links and one-line descriptions. llms-full.txt is the territory: the actual content of those pages inlined into one file, so a model can ingest everything in a single request. Use llms.txt as the lightweight index every site can maintain by hand. Use llms-full.txt when you want a model to hold your full documentation in one paste, which suits docs sites and APIs more than a personal site or blog.
How to write an llms.txt file
The structure is fixed and short:
- Start with an H1: the name of your site or project.
- Add a blockquote summary: one or two sentences on what the site is and who it is for.
- Group links under H2 headings: such as Key resources, Guides, or Products. Each link is a Markdown link followed by a colon and a short description of what the page covers.
- Keep it curated: list the pages you actually want cited, not every URL. This is an editorial file, not a sitemap.
- Host it at the root: /llms.txt, served as plain text, alongside your robots.txt and sitemap.
A minimal version for a personal site is an H1 name, a one-line summary, and three sections: about, key writing, and contact. That is enough to start, and you grow it as you publish.
Is llms.txt the same as robots.txt or a sitemap?
No. robots.txt tells crawlers what they are allowed to fetch, and a sitemap lists every URL for search indexing. llms.txt does neither. It is a curated, human-written guide to your best content, formatted for a language model to read cheaply. robots.txt is about permission, the sitemap is about coverage, and llms.txt is about comprehension. They sit side by side, and one does not replace another.
Does llms.txt actually work?
Honestly, the jury is still out. As of 2026 no major AI provider has publicly confirmed that it reads llms.txt to decide what to cite or how to answer, and Google has said it does not use it for search. So treat it as a low-cost bet, not a proven ranking lever. The upside is real but modest: it is quick to add, it forces you to name your most important pages, and some AI tools and crawlers do fetch it. The downside is only the few minutes it takes to write. For a content site already investing in generative engine optimisation and the shift from SEO to AEO and GEO, it is worth shipping, with clear eyes about what it does and does not guarantee.
FAQ
What is an llms.txt file used for?
An llms.txt file gives AI models a curated, Markdown map of your most important pages so they can find and understand your content cheaply, without parsing full HTML. It is meant to improve how large language models and AI search tools read and represent your site.
Where do you put llms.txt?
At the root of your domain, served as plain text at /llms.txt, in the same place as robots.txt and your sitemap. Some sites also publish an expanded /llms-full.txt with the full content of those pages inlined for single-request ingestion.
Is llms.txt the same as robots.txt?
No. robots.txt controls which URLs crawlers may fetch. llms.txt is a curated, human-written guide to your best content, written in Markdown for language models to read. One governs permission, the other aids comprehension, so they coexist rather than replace each other.
Does Google use llms.txt?
As of 2026, no. Google has said it does not use llms.txt for search ranking, and no major AI provider has confirmed using it for citations. It remains a low-cost, unproven convention. Add it for the small upside and the clarity it forces, not because it guarantees visibility.
The honest summary: llms.txt is a sensible, cheap habit, not a magic switch. Write a tight, curated one, keep it current as you publish, and pair it with the structural work that AI engines clearly do reward: clean schema, answer-first content, and a fast site. The map helps only if the territory is worth visiting.