What the file is
Proposed by Jeremy Howard (Answer.AI) in 2024, llms.txt is a plain-markdown file listing what your site is and which pages matter, so a language model with a small context window can orient itself without crawling everything. Think of it as a curated sitemap written for machines that read prose.
It is NOT robots.txt (which controls crawler ACCESS) and not sitemap.xml (which lists URLs for indexing). llms.txt is purely informational — it grants nothing and blocks nothing.
The honest evidence check
- No major AI provider (OpenAI, Anthropic, Google, Perplexity) has publicly committed to consuming llms.txt as of mid-2026.
- Server-log studies across the industry show AI crawlers requesting it rarely — occasional fetches exist, systematic use doesn't.
- Adoption is real on the publishing side (many dev-tool and SaaS sites ship one), which keeps the proposal alive — conventions sometimes win by ubiquity.
So the honest framing: llms.txt is a lottery ticket that costs five minutes. Sensible to hold, silly to build a strategy on. What verifiably moves AI visibility is the content itself — direct answers, sourced claims — and your presence in the sources engines already read.
If you add one, do it right
- One H1 with your site name, a one-paragraph plain-language description, then a short annotated list of your most important pages.
- Write it like an elevator pitch for a machine: what you do, for whom, and where the proof lives.
- Keep it current — a stale llms.txt describing a product you've pivoted away from is worse than none.
- Don't duplicate your whole sitemap; curation is the point.
Promvia's GEO audit checks for llms.txt and generates one from your site's data — weighted honestly in the score as the minor signal it is, alongside the checks that matter more (AI-crawler access, answer-shaped content, structured data).
Frequently asked questions
Will llms.txt improve my ChatGPT visibility?
There's no evidence it does today. It might become useful if engines adopt it. Add it for the option value, then spend your real effort on quotable content and third-party presence.
Is llms.txt the same as blocking or allowing AI bots?
No — access control lives in robots.txt (and your CDN's bot settings). llms.txt has no enforcement meaning at all.
Should agencies ship llms.txt for clients?
Sure — as a five-minute checklist item, honestly labeled. Charging for it as a headline 'AI optimization' deliverable is the kind of oversell that burns client trust.