llms.txt: a field guide to the newest file in your web root

There is a new file quietly appearing in web roots alongside robots.txt and sitemap.xml. It is called llms.txt, it is written in plain markdown, and its premise is disarmingly simple: if language models are going to read your website, you may as well hand them a well-written summary instead of hoping they assemble one from your navigation menu. Whether the big AI vendors actually read it yet is a fair question, and we will answer it honestly.

We ship one on oxwynstudio.com. This is what the convention actually says, where it fits among the files you already have, and the unglamorous case for adopting it before the verdict is in.

What llms.txt actually is

llms.txt is a proposed convention, published in September 2024 by Jeremy Howard, the co-founder of Answer.AI and fast.ai, and documented at llmstxt.org. The pitch: HTML pages are a hostile format for language models. They arrive wrapped in navigation, cookie banners, scripts and boilerplate, and a model with a limited context window (the amount of text it can consider at once) burns most of its budget parsing furniture. So you provide a single markdown file at /llms.txt that says, in curated human-written prose, what the site is and where the important things live.

The format is deliberately minimal, and it is markdown rather than a machine syntax on purpose, since the intended reader is a language model:

• An H1 with the site or project name (the only required element).
• A blockquote summary: one short paragraph saying what this site is.
• Optional H2 sections containing link lists, each link with a one-line description.
• An optional section literally titled "Optional" for links a model can skip when context is tight.

The original proposal also suggests a companion pattern: markdown versions of key pages at the same URL with .md appended, so a model that wants depth can fetch clean text instead of HTML. We do exactly this for our service pages, so /services/secure-web-development.md serves the prose without the chrome.

That is the whole convention. No registry, no validation authority, no standards body. As of mid 2026 it remains a community proposal, not an IETF or W3C standard, and the spec is still maintained by the community around the original authors.

robots.txt, sitemap.xml, llms.txt: three files, three jobs

The easiest way to misuse llms.txt is to confuse it with its older neighbours, so it is worth being precise:

• robots.txt is permission. It tells crawlers, by user-agent name, what they may fetch. It says nothing about what anything means. It is the bouncer.
• sitemap.xml is inventory. An exhaustive machine-readable list of every URL you want indexed, with modification dates. No curation, no meaning. It is the warehouse manifest.
• llms.txt is orientation. A curated, opinionated summary for a reader with limited attention: here is who we are, here are the dozen pages that matter, here is one line on why each matters. It is the front-of-house briefing.

Nothing belongs in two of these files. Access rules stay in robots.txt (an llms.txt cannot block anything, and was never meant to). Exhaustive URL listings stay in the sitemap; an llms.txt that lists two hundred links has missed the point of curation. And the summary prose in llms.txt belongs nowhere else, because neither of the older formats has anywhere to put a sentence.

The same logic explains what belongs inside ours: services with one-line descriptions, plain definitions of the concepts we get asked about (what AEO and GEO actually mean, what Website-as-a-Service includes), public pricing, and a contact link. The things a model would need to describe us accurately to a stranger.

The honest adoption picture

Now the part most articles on this subject fudge. Has llms.txt won? No, and anyone telling you it is a ranking factor is ahead of the evidence.

What is true, as of mid 2026: adoption among developer-facing companies is real and visible. Anthropic, Stripe, Cloudflare, Vercel, Supabase and others publish llms.txt files, documentation platforms generate them automatically, and surveys put deployment somewhere in the low single-digit percentages of sites, concentrated heavily in technical products. It has moved from curiosity to a normal, if minority, practice.

What is not established: confirmed consumption by the major AI vendors. No large provider has publicly committed to fetching llms.txt as part of answering or training, Google's search team has been openly dismissive of it, and independent log analyses disagree about how often AI crawlers request the file at all. The tooling ecosystem (generators, validators, directories) is currently more mature than the demand side. That is the truthful state of play: a convention with genuine grassroots adoption and no guaranteed reader.

If that sounds like a reason not to bother, hold that thought until the last section.

What ours looks like

Here is a trimmed version of the file live at oxwynstudio.com/llms.txt, showing the shape rather than the full contents:

# Oxwyn Studio

> UK-headquartered digital engineering studio delivering bespoke,
> hand-coded websites, custom web apps, SaaS, ecommerce and applied
> AI for organisations across the UK, USA and Europe.

## Services

- [Website-as-a-Service](https://oxwynstudio.com/website-as-a-service):
  Subscription websites with hosting, maintenance and ongoing
  evolution from one predictable monthly fee.
- [Secure Web Development](https://oxwynstudio.com/services/secure-web-development):
  Hand-coded, security-hardened Next.js websites with strict CSP
  and OWASP Top 10 coverage. Markdown:
  https://oxwynstudio.com/services/secure-web-development.md

## Key concepts

- AEO (Answer Engine Optimisation): being the cited source inside
  AI answers from ChatGPT, Perplexity and Google AI Overviews.
- WaaS (Website-as-a-Service): a website delivered as a
  subscription, with build, hosting, maintenance and evolution
  bundled into one monthly fee.

## Pricing

- Website-as-a-Service: from £99/mo plus a one-off setup fee.

## Contact

- Book a call: https://oxwynstudio.com/contact

A few deliberate choices worth stealing. Every link carries a description, because the description is the payload; a bare URL tells a model nothing it could not get from the sitemap. The key concepts section defines our own vocabulary, so a model answering "what is WaaS" has our definition available rather than improvising one. Pricing is stated because it is public anyway and models get asked about it. And the whole file reads like something a competent colleague would write, because that is the intended register: this file is documentation for a reader, not markup for a parser.

Maintenance matters more than authorship. An llms.txt that still lists last year's services is worse than none, because you have curated your own misrepresentation. Ours is reviewed whenever the services line-up changes, which is a one-minute diff precisely because the file is short.

Why we ship one anyway

Given the honest adoption picture, why bother? Three reasons, none of which require faith.

It is nearly free. Ours took under an hour to write from copy that already existed, deploys as a static file, and costs nothing to serve. Against that outlay, even a modest probability that AI retrieval systems consume it (some smaller tools already do, and agents fetching sites can be pointed at it today) is a trade worth taking. This is a lottery ticket priced at loose change.

It is harmless if ignored. llms.txt grants no permissions, blocks no crawlers, and duplicates nothing that could drift into contradiction if you keep it curated. The failure mode of shipping one into an indifferent world is that a tidy markdown file sits unread. Compare that with the failure mode of most speculative optimisations, which is technical debt, and it is an easy call.

The direction of travel is one way. Whatever happens to this specific filename, the underlying need is not going away: machine readers with limited context will keep wanting curated, clean-text orientation, and sites that can provide it will be easier to represent accurately than sites that cannot. Writing an llms.txt forces you to produce that artefact. If a successor convention wins instead, the content ports in minutes.

What we will not do is dress this up as more than it is. Shipping llms.txt will not get you cited by ChatGPT next week, and nobody can guarantee AI citation with this file or any other technique; that caveat is house doctrine and it applies here with full force. This is a low-cost, forward-looking hedge that also happens to sharpen your own copy. We ship it as part of the same discovery layer as robots.txt, sitemap.xml and structured data on every Website-as-a-Service build, and treat all four as one system rather than four tricks.

Key takeaways

• llms.txt is a markdown file at your web root, proposed by Jeremy Howard of Answer.AI in September 2024, giving language models a curated summary of your site with described links.
• It complements rather than replaces its neighbours: robots.txt governs access, sitemap.xml lists inventory, llms.txt provides orientation and meaning.
• Adoption is real but early, concentrated in developer-facing companies, and no major AI vendor has confirmed consuming the file; treat any stronger claim with suspicion.
• A good llms.txt is short, curated and maintained, with a one-line description on every link and definitions of your own vocabulary.
• The case for shipping one is expected value, not evidence of payoff: near-zero cost, zero harm if ignored, and content that ports to whatever convention wins.

Frequently asked

Does llms.txt replace robots.txt or sitemap.xml?

No. It has no access-control power and no indexing role, so both older files keep their jobs. llms.txt adds the one thing neither can express: curated prose about what your site is and which pages matter, written for a reader rather than a parser.

Do ChatGPT, Claude or Gemini actually read llms.txt?

None of the major vendors has publicly confirmed using it as of mid 2026, and log evidence is mixed. Some smaller AI tools and agents do fetch it, and documentation platforms have adopted it widely. Honest answer: consumption is unproven, which is why we frame it as a cheap hedge rather than a tactic with measurable returns.

What should a small business put in an llms.txt?

An H1 with your name, a two-sentence blockquote saying what you do and for whom, a short list of your most important pages with one-line descriptions, definitions of any terms customers ask about, and public pricing if you have it. Keep it under a page and review it whenever your offering changes.

Will llms.txt improve my AI visibility or citations?

Nobody can guarantee that, and no current evidence ties the file to citation rates. Its realistic value is making your site easier to represent accurately if and when systems consume it, at a cost so low the bet needs almost nothing to pay off. Structure improves interpretation; citation remains outside anyone's control.