llms.txt

llms.txt is a proposed convention for publishing an LLM-friendly index of a website or documentation set. It is usually served at /llms.txt and written in Markdown so AI assistants, coding agents, and retrieval systems can quickly discover the most relevant pages.

The file typically contains:

• a site or project title;
• a short description;
• grouped links to important Markdown or documentation pages;
• brief descriptions of each link; and
• optional references to larger bundles such as llms-full.txt.

The goal is different from robots.txt or sitemap.xml. A sitemap helps crawlers enumerate URLs. robots.txt communicates crawl permissions. llms.txt helps an AI system decide what content is useful to read for inference-time context.

For developer documentation, llms.txt can reduce wasted crawling and improve context engineering by pointing agents to canonical Markdown, API references, examples, and setup guides. It is especially useful for coding agents that need accurate product documentation.

llms.txt is not an access-control mechanism. Sensitive or private content must still be protected by authentication and authorization. The file should only link to content the publisher intends AI tools to read.

The original Answer.AI llms.txt proposal describes the format as a way to help language models use a website at inference time.