# llms.txt

> VI: https://5ac.vn/wiki/llms-txt/ · EN: https://5ac.vn/en/wiki/llms-txt/ · lang: en · spec_as_of 2026-10-18 (spec v2) · cluster C11

A markdown file at the site root that packages background notes and links to clean text pages, so an AI agent can read before it uses your content.

---
## Definition

llms.txt is Jeremy Howard's open proposal for a markdown file served at a site's `/llms.txt` path. It carries a short introduction plus links to trimmed, model-readable pages, so an agent reads the map first and fetches only the documents it needs, instead of re-parsing the site's full HTML.

## Mechanism

The proposal has two halves. First, the llms.txt file itself: it can sit at the site root or at any path, and one file describes every page under that path; when several files apply, agents should use the most specific one. Second, each page an agent may need should publish a clean markdown version at the same URL, with `.md` appended or substituted (`page.html.md`, `page.md`, or `index.md`). To make those discoverable, the spec recommends two standard link relations: `rel="alternate" type="text/markdown"` pointing at a page's markdown twin, and `rel="describedby"` pointing at the llms.txt file covering it, declared either as HTML `<link>` elements or as an HTTP `Link:` header, which also works for non-HTML resources.

The file format uses markdown as its own schema, in a fixed order:

| Part | Role | Required |
| --- | --- | --- |
| Optional BOM | byte-order mark at the top | no |
| H1 | project or site name | yes — the only required part |
| Blockquote | short summary with the background needed to read the rest | no |
| Free markdown sections (no headings) | notes on how to interpret the listed files | no |
| H2 sections with "file lists" | each item is a `[name](url)` link, optionally followed by `:` and a note | no |
| "Optional" section | convention for secondary links an agent may skip when context is tight | no |

The design goal is token cost. Context windows grew but still cannot hold a whole website, and every wasted token is paid for in latency and money. The llms.txt stays small; the detail lives behind the links and is fetched only when needed. The spec also positions itself next to existing standards: robots.txt "lets automated tools know what access to a site is considered acceptable", sitemap.xml lists every indexable page for search engines, while llms.txt is a curated, context-rich map used on demand — the page records that it has in practice served inference rather than training.

As for adoption, the v2 page (accessed 2026-10-18) reports thousands of sites publishing an llms.txt, documentation platforms (Mintlify, GitBook) and CMSs (Yoast SEO, AIOSEO, Wix) generating one automatically, Chrome's Lighthouse auditing for it among its agentic-browsing checks, and AI labs (OpenAI, Anthropic, Gemini) publishing llms.txt files for their own developer docs.

Flow: Agent reads /llms.txt → picks the relevant section → fetches the .md page → answers from what it just read

## Spec anchor

The proposal page is titled "The /llms.txt file, v2", published 2024-09-03 and updated 2026-08-10. Its legal status deserves a blunt statement: llms.txt remains an informal proposal, "open for community input", not owned by any standards body. Maintenance lives in the AnswerDotAI/llms-txt GitHub repository (Apache 2.0), used for version control and public discussion.

Sources fetched for this entry:

- <https://llmstxt.org/> — the full v2 proposal text — accessed 2026-10-18
- <https://github.com/AnswerDotAI/llms-txt> — the proposal's repository; README mirrors the spec page — accessed 2026-10-18
- <https://5ac.vn/llms.txt> — this site's own llms.txt, HTTP 200 — accessed 2026-10-18

## Academic anchors

The scholarly anchor here pins the measurement side: *AX is the New AEO* (arXiv:2609.34951, Ido Finder et al., submitted 28 Sep 2026) runs *"37,927 agent journeys"* over *"1,056 real businesses"* across four independent harnesses and reports that *"Agent-ready businesses have answers built from their own pages 78% of the time against 56%"*. That is precisely the lever llms.txt pulls: when an agent can fetch and read the primary page, the answer is built from that page rather than from third-party sources — the paper names the property *"agent experience (AX)"* and concludes *"being readable beats being talked about"*.

## FAQ

**Does llms.txt replace sitemap.xml?** No. A sitemap enumerates every indexable page for search engines; llms.txt is a curated, context-bearing map pointing at LLM-readable versions — something sitemaps never describe. Both coexist.

**Must an agent read every llms.txt that covers a page?** The spec's rule: one file covers all URLs under its path, and where several apply the agent should use the most specific. `/docs/llms.txt` wins over `/llms.txt` for pages under `/docs/`.

**Is the `.md` twin page mandatory?** Not mandatory, but it is the second half of the proposal: links inside llms.txt should point at LLM-friendly content, and the same-URL `.md` page is the recommended form. The index file itself must stay small enough to fit in context.

**How is this different from robots.txt?** robots.txt governs whether automated tools may access content at all (search indexing, scraping policy); llms.txt grants nothing — it supplies background at the moment an agent needs it to help a user.

**Does 5ac.vn practice what it documents?** Yes. This site publishes its own llms.txt at the root (checked live, HTTP 200, on 2026-10-18), and this wiki is part of the same agent-serving strategy.

## Related

- [A New Era for Websites: When Your First Visitor Isn't Human](/en/blog/agentic-era-websites-that-serve-agents/)
- [Local SEO for AI Overviews: Vietnamese businesses](/en/blog/local-seo-ai-overviews-vietnamese-businesses/)
- [What is the Model Context Protocol (MCP)?](/en/wiki/mcp/)
- [What is Agent2Agent (A2A)?](/en/wiki/a2a/)