The /llms.txt file

The /llms.txt proposal, now in its second version, asks websites to publish a markdown file at /llms.txt that gives LLMs a curated overview and links to clean markdown versions of key pages. The author, Jeremy Howard, describes it as a response to agents reading HTML pages that are cluttered with navigation, ads, and JavaScript, and says thousands of sites now publish the file. The proposal also recommends serving each page as markdown at the same URL with .md appended and linking the two using standard link relations. It positions llms.txt as complementary to robots.txt and sitemap.xml rather than a replacement for them.