What Is llms.txt and Does Your Site Need One?

llms.txt is a plain Markdown file at the root of a website that offers AI systems a short map of its key pages. For getting named by ChatGPT, Claude or Gemini, most sites do not need one: server logs from 137,000 domains show that 97% of these files were never requested, and a study of 300,000 domains found no link to AI citations.

This guide explains what the file is, what the measurements say, where it still has a use, and what to work on instead if your goal is to be recommended by AI assistants.

What is llms.txt in simple terms?

llms.txt is a text file written in Markdown that sits at a fixed address: yoursite.com/llms.txt. The file opens with a one-sentence summary of what the site is about, then lists links to its most useful pages, each with a short description. Jeremy Howard, co-founder of Answer.AI, proposed the format in September 2024, and the specification lives at llmstxt.org.

The idea behind llms.txt is sound on paper. An AI model reads a site through a limited context window, and menus, cookie banners and layout code waste that budget. A clean Markdown summary is cheap to read. The catch is that the idea only works if the AI systems agree to fetch the file, and the major assistants have not agreed.

What does an llms.txt file look like?

An llms.txt file has four parts: a Markdown H1 with the site name, a blockquote with a one-line summary, H2 sections that group related links, and the links themselves. Here is a complete, valid example for an imaginary bakery.

# Sunrise Bakery

> Sunrise Bakery is a family bakery in Austin, TX. We bake sourdough daily,
> deliver locally, and run weekend baking classes.

## Products

- [Sourdough menu](https://sunrise-bakery.com/breads.md): Daily breads and prices
- [Wedding cakes](https://sunrise-bakery.com/cakes.md): Custom orders and lead times

## Optional

- [Press](https://sunrise-bakery.com/press.md): Reviews and awards

The Optional section is part of the specification: it marks links a reader may skip when its budget is tight. The specification also suggests linking to .md versions of pages, because Markdown is easier for a model to digest than rendered HTML.

Does anyone actually read llms.txt?

Publishing is common; reading is rare. Ahrefs checked the server logs of 137,000 domains for May 2026 and found that about 38,000 of them published a valid llms.txt, yet 97% of those files received no requests at all. Most of the requests that did arrive came from SEO audit tools and other non-AI bots. Retrieval bots linked to ChatGPT and Perplexity made up about 1%.

No major AI provider has said its assistant uses llms.txt when answering questions. OpenAI's crawler documentation describes GPTBot, OAI-SearchBot and ChatGPT-User and how to control them through robots.txt, and it does not mention llms.txt. Anthropic and Perplexity have made no public statement that they read the file on other people's sites.

Google is explicit. Its guide to generative AI features in Search says that sites do not need new machine-readable files or AI text files to appear in Search, including its AI features, because Google Search does not use them.

Does llms.txt help you get cited by ChatGPT or Claude?

No measurement so far shows that it does. SE Ranking compared about 300,000 domains and found no relationship between having llms.txt and how often a domain is cited in major AI answers. In their machine-learning model, removing the llms.txt feature made the predictions more accurate, which is what happens when a variable carries no signal.

The two findings fit together. A file that assistants rarely fetch cannot shape what they say. For a business that wants to be named when a buyer asks an AI assistant for a recommendation, llms.txt is not a lever, and anyone selling it as one is selling hope.

How is llms.txt different from robots.txt, sitemap.xml and Schema markup?

robots.txt controls which pages crawlers may fetch, sitemap.xml lists pages for search engines, Schema.org markup describes things inside a page, and llms.txt proposes a curated summary for AI readers. The first two are settled infrastructure that AI crawlers respect. The table below compares all four by what the evidence says each one does for AI answers.

The practical reading of the table: robots.txt is the one file here that can switch AI visibility off, so check it first. Schema markup is still worth having for Google rich results and for naming your company unambiguously, and our guide to schema markup covers which types matter. llms.txt is at the bottom of the list.

What is llms.txt actually good for?

llms.txt has a real use in software documentation. AI coding assistants and agents can load a whole manual through one clean file, which is why developer platforms publish one: Anthropic maintains an llms.txt for its own developer docs. If your product has an API or technical documentation that developers read through AI tools, the file is worth an hour.

For a shop, a clinic, an agency or a SaaS marketing site, the honest answer is different. The file will most likely never be read, and the hour is better spent on the work in the next section.

What makes AI assistants name a business instead?

The factors with the strongest evidence behind them are less exotic than a new file:

• A crawler has to be able to read the page. That means robots.txt does not block AI user agents, and the page carries its content in the HTML the server returns, because most AI crawlers never execute JavaScript.
• The page has to be findable. AI search products lean on web search results, so a page that is indexed and ranks for the question has a chance to be retrieved.
• Other sites have to mention the business. Across 75,000 brands, Ahrefs found that mentions of a brand on the web correlate with its visibility in Google AI Overviews about three times more strongly than backlinks do. It is a correlation, not proof of cause, but no other factor in that study came close.
• The facts have to be in the visible text. Prices, plans, locations and direct answers to buyer questions should appear on the page itself. Chat assistants read that text, not the markup behind it, and an Ahrefs test of 1,885 pages that added JSON-LD found no meaningful change in AI citations.

If you still want an llms.txt, how do you create one?

A documentation site or an API product can write a solid llms.txt by hand in under an hour:

1. Pick the pages that explain the product best, such as the quick start, the API reference and the pricing page.
2. Write one accurate sentence about what the product is and put it in the blockquote at the top.
3. Group the links under H2 sections and give each link a short description after a colon.
4. Save the file as plain text named llms.txt and serve it from the root of the domain, next to robots.txt.
5. Open yoursite.com/llms.txt in a browser to confirm it returns the file with a 200 status.

The specification at llmstxt.org is short and is the reference for anything beyond these steps.

Frequently asked questions

Does llms.txt help SEO?

No. Google has said that Search does not use llms.txt, so the file has no effect on rankings. Titles, content, internal links and technical health still do the SEO work.

Does llms.txt block AI crawlers?

No. llms.txt cannot forbid any company to crawl a site or train on its content. Blocking a crawler is the job of robots.txt and its user-agent rules for bots like GPTBot or ClaudeBot.

What is llms-full.txt?

llms-full.txt comes from the same specification. Instead of linking out to pages, it holds the full text of the content in one file, so a coding assistant can load an entire manual in a single request. Documentation sites are the ones that use it.

Does AI recommend you or your competitor?

Whether an assistant names you is a fact you can check rather than guess. When a buyer asks ChatGPT or Claude for the best option in your market, the answer either includes your business or it does not.

The free check puts real questions from your market to the AI assistants, shows which brands they name, and compares what they say about you with your own site. It takes about three minutes. For the manual version, how to check if AI mentions your brand walks through ten questions and a spreadsheet.