What Is llms.txt — and Does Your Business Need One?

Quick primer

llms.txt is a lightweight Markdown file you drop in your site’s root to tell large language models what your site is about — think robots.txt for AI. The specification was proposed by Jeremy Howard of fast.ai/Answer.AI in September 2024 and lives at llmstxt.org. It contains a short title, grouped headings, links to key pages with one-line blurbs, and optionally a companion llms-full.txt with full page content. The pitch is simple: give AI a tidy roadmap so it doesn’t have to crawl every nook and cranny.

Who’s using it — and does it work?

It has been reported that tools and documentation-heavy sites are the early adopters: Anthropic, Cloudflare, Stripe, Zapier, and Supabase are among those maintaining llms.txt files. It has also been reported that BuiltWith tracked over 844,000 sites with the file implemented. But don’t expect magic overnight. Google’s John Mueller reportedly called the spec “unnecessary,” and it has been reported that no AI platform has formally made llms.txt a ranking or citation factor. On the flip side, it has been reported that Microsoft, OpenAI, and other AI firms are crawling and indexing these files, according to Mintlify — so the standard is gaining traffic if not definitive evidence yet.

Should your business create one?

Short answer: maybe. If you run a content-heavy site, product docs, or a blog with dozens of posts, a ten-minute llms.txt is low-effort insurance. Why not? It costs nothing and could pay future dividends if models start trusting the file’s structure. If your site is a five-page brochure — don’t sweat it. Your time is better spent on core SEO basics: quality content, reviews, and a well-optimized Google Business Profile. Ask yourself: is this a tiny hedge against future AI discovery or a distraction? For many, it’s the former.

Bottom line

AI search is evolving fast and standards are still forming. llms.txt isn’t a proven ranking lever today, but it’s an inexpensive, reversible signal to feed into ­emerging AI systems. Spend ten minutes if you have the content depth; otherwise, keep your head down on the fundamentals. After all, a small bet now might turn into an early-mover advantage later — or just a neat little file you never miss.

Sources: semarkglobal.com, Hacker News