llms.txt

llms.txt is a proposed convention: a markdown file served at the root of a site that gives language models a curated guide to its most useful content.

It exists because a model reading an ordinary page has to work past navigation, scripts, and markup to reach the substance. The file lists the pages worth reading with a line on each, and a companion llms-full.txt can carry the content itself. It is a convention rather than a ratified standard, so support varies by crawler.

Closely related concepts include Training Data Poisoning, Foundation Model, Generative AI, Large Language Model (LLM). Each is defined in the Prefactor glossary.