Every year or two, the SEO industry produces a new magic file that promises to fix visibility overnight. In 2026, that file is llms.txt — a proposed standard that has clients asking us the same question weekly: do we need one? The short answer is that it probably won't hurt you, it probably won't help you much either, and the time you spend debating it is better spent on the things AI systems demonstrably do read.
The numbers explain the confusion. A SE Ranking study of 300,000 domains found llms.txt adoption has reached roughly one in ten websites. Yet crawl-log analyses tell the opposite story: in one 90-day window covering more than 500 million AI bot visits, only a few hundred requests targeted llms.txt directly. Widespread adoption, negligible consumption. That gap is worth understanding before you invest in it.
llms.txt is a plain Markdown file placed at the root of your domain that offers AI systems a curated map of your most important content — a condensed, machine-friendly summary of what your site covers and where the good pages live. The key distinction: robots.txt restricts access, while llms.txt invites and guides understanding. It's an open invitation, not a gate. It costs perhaps half an hour to create, which is exactly why so many sites have added one.
Here's the uncomfortable part. Google has said plainly that it doesn't support llms.txt and isn't planning to — one Google search engineer compared it to the long-discredited keywords meta tag. The major answer engines behave the same way in practice: they crawl your actual pages. What gets your content into an AI answer is the rendered HTML itself — clear heading structure, direct answers placed near the top of the page, valid schema markup, consistent business information, and internal links that establish which pages matter.
There is one context where the file earns its keep: documentation-heavy sites. AI coding assistants genuinely benefit from a short, curated index of technical pages, and developer-tool companies report real usage there. If you publish extensive docs, add one. For everyone else, treat it as a low-cost experiment — reasonable to run, unreasonable to prioritise over proven technical SEO work.
llms.txt is a business card handed to a visitor who rarely asks for one. Make sure the building itself is worth walking into first.
The work that gets brands cited by AI Overviews, ChatGPT, and Perplexity in 2026 looks suspiciously like the work that always earned rankings: publish genuinely useful content on topics you can claim authority over, structure it so a machine can lift an answer cleanly, mark it up with schema, earn third-party mentions that corroborate your expertise, and keep the technical foundation fast and crawlable. Answer engines cite sources they can parse and trust. No root-directory text file substitutes for either quality.
Add an llms.txt file if it closes the debate — it takes half an hour and carries no penalty. But file it mentally next to the keywords meta tag: a signal almost nobody reads. Your AI visibility budget belongs in structured, citable, authoritative content on a technically sound site. That's the strategy the crawler logs actually support.
We audit how search engines and AI assistants crawl, read, and cite your site — then fix what's keeping you invisible.
Talk to us