llms.txt explained
An llms.txt file offers a curated map of a site for clients that choose to read it. Learn its structure, discovery links and limits.
llms.txt is a plain text file that tells AI agents and language models what a site contains and where the important content lives. It sits at /llms.txt at the root of a site, or at any path inside it, in which case it covers the pages under that path and the most specific file applies. It works like a guide written for machines. A human reads the rendered page, an agent reads llms.txt and follows the links it lists.
File structure
The file opens with the site name as an H1, then a short summary as a blockquote, then the key pages and resources as markdown links grouped under H2 headings. Only the H1 is required. Everything after it, the summary, the headings, and the grouped links, is recommended rather than mandatory, though the format expects each item in a file list to be a markdown hyperlink, and a minimal file with just the name is still a valid one. Some sites also publish llms-full.txt, a single file that bundles the full text of the site so an agent can read everything in one request instead of crawling many pages. The llms.txt of turva.dev departs from the file-list format on purpose and lists its agent endpoints as plain addresses rather than links, because two of them answer a plain GET with something other than 200 by design, the MCP endpoint with 405 and the x402 endpoint with 402, and the whole group keeps one plain format instead of mixing linked and unlinked lines within it. Its pricing, business details, engagement model and contact sections also carry plain values rather than page addresses, so those lines were never links to begin with.
The proposal reached v2 in August 2026 and the file format did not change. What changed is how an agent finds the machine-readable forms. A page now names them with two standard link relations, rel="alternate" type="text/markdown" for the markdown version of the page and rel="describedby" for the llms.txt that covers it, given either as HTML link elements or as an HTTP Link header. v2 also accepts both address forms for a markdown version, page.html.md and page.md, and it drops the context expansion tooling that v1 described, so the Optional section is a convention for secondary links and carries no mechanical meaning any more.
How clients find it
A client finds llms.txt at the root of the site, /llms.txt, or through the rel="describedby" link a page names, and it finds the markdown version of a page through rel="alternate" or by asking for text/markdown. The file is worth publishing for cost and clarity. A normal HTML page carries navigation, scripts, and styling that an agent has to wade through, and that spends tokens and invites mistakes. An llms.txt file, paired with markdown content negotiation, lets an agent fetch a clean text version and skip the noise. On turva.dev the markdown version of a page costs a fraction of the HTML, which lowers the chance that a long page runs past a reading tool's content limit and arrives cut short. turva.dev publishes llms.txt and llms-full.txt, serves markdown on request, and publishes the markdown version of every page at its own .md address with both v2 link relations. Whether a client fetches any of it depends on the client. A clear llms.txt serves the clients that read it, and no assistant is obliged to be one of them.
What it does not replace
llms.txt is not a ranking trick and it does not replace a sitemap or robots.txt. A sitemap lists the URLs a site wants crawlers to find, the canonical ones it prefers when the same content sits at more than one address, not necessarily every URL that exists. robots.txt sets crawl rules. llms.txt is a curated, human-written map of what matters, aimed at models. The three work together. Google states that Search, including its generative features, ignores the file, so publish it for the clients that fetch it rather than as a route into Google or into search rankings.
Validate your file
A validator reads the file the way a client would. It checks that the H1 is present, that each link is a well-formed absolute address and that the grouping under each heading is well formed. The free validator on this site does not fetch the pages the links point to, so request each one once before publishing to confirm that it still answers.
Check any site's llms.txt structure with the free validator, which reads the file and the home page without signing up.
Frequently asked
What is llms.txt?
llms.txt is a plain text file that tells AI agents and language models what a site contains and where the important content lives. It sits at the root of a site or at any path inside it, where it covers the pages under that path. It does not replace robots.txt or a sitemap, and both keep their own jobs.
Does llms.txt help with search ranking?
No. It is not a ranking trick. Google states that the file neither harms nor helps visibility or rankings in Search, because Search ignores it. What llms.txt gives is a curated map of the content to those clients that do fetch it.
What does an llms.txt file contain?
The site name and a short summary, then the key pages and resources as markdown links, often grouped under headings. Some sites also publish llms-full.txt, which bundles the full text so an agent can read everything in one request.
What changed in v2 of llms.txt?
The file format did not change. v2 added two standard link relations so an agent finds a page's markdown version and its llms.txt without guessing, accepted page.md alongside page.html.md as the address of a markdown version, defined what an llms.txt in a subpath covers, and dropped the context expansion tooling along with the mechanical meaning of the Optional section.