The direct answer

The llms.txt file is a plain text document placed in your website’s root directory that gives AI language models a curated, structured index of your most important pages. It functions like a table of contents written specifically for AI systems — telling them what your site covers, who it serves, and which pages are most important for understanding its content. A well-configured llms.txt file accelerates AI indexing, reduces crawling errors, and helps AI systems build a more accurate model of your site’s purpose and content.

How llms.txt differs from robots.txt and sitemaps

Your robots.txt file tells crawlers what they are permitted to access. Your XML sitemap tells crawlers what pages exist. Your llms.txt file tells AI language models what your site is about and which pages to prioritize for understanding that purpose. These three files serve different functions and all three should be correctly configured for complete AI accessibility.

Robots.txt and sitemaps are primarily about access and discovery. llms.txt is about understanding and prioritization. An AI system that reads your llms.txt file before crawling your site arrives with context — it knows what type of site this is, what subject it covers, who the audience is, and which pages are the most important — rather than having to infer all of this from scratch through crawling.

The llms.txt file format

The llms.txt format uses a simple Markdown-style structure. Here is the complete format:

# Site Name

> One sentence describing what your site covers and who it serves.

## Core Pages

- [Page Title](https://yoursite.com/page): Brief description of what this page covers

- [Page Title](https://yoursite.com/page): Brief description of what this page covers

## Topic Areas

- [Topic Name](https://yoursite.com/topic): Brief description of this topic cluster

## About

- [About](https://yoursite.com/about): About this site and the author

## Optional: External Links

- [Reference](https://external.com): A key external resource you cite or reference

What to include in your llms.txt

The header section (required)

The H1 line is your site name exactly as it appears in your Organization schema. The blockquote (the line starting with >) is a single sentence describing what your site covers and who it serves. This is the context AI systems read first — make it specific and accurate. “A complete guide to AEO, GEO, SEO, ANI, ASI, and Lead Funnels for WordPress site owners using free tools” is better than “digital marketing resources.”

Core Pages (required)

List your most important pages in the Core Pages section. These should be your pillar pages — the comprehensive overview pages for each major topic area on your site. Include a brief but specific description for each. The description should state what the page covers in a way that helps an AI system understand its scope and purpose.

Topic Areas (recommended)

List your content clusters in the Topic Areas section. This helps AI systems understand the breadth and organization of your content — that your site covers multiple related topics in depth rather than being a single-topic site with supporting content.

About (required)

Always include a link to your About page. This connects your llms.txt entity description to your Person schema and author bio — creating a coherent entity signal that reinforces both the site’s purpose and the author’s credentials.

How to create and deploy your llms.txt

1. Create a plain text file named exactly llms.txt — lowercase, no spaces
2. Write the content following the format above
3. Save it as plain text (UTF-8 encoding)
4. Upload it to your site’s root directory via your hosting file manager or FTP
5. Verify it is accessible by visiting yoursite.com/llms.txt in your browser — you should see the plain text content

Does llms.txt actually work?

The llms.txt standard is relatively new and its adoption varies by AI system. Perplexity has confirmed reading llms.txt files when available. OpenAI has indicated interest in the standard. Google has not formally endorsed it but it does not conflict with any Google guidance and may be read by Google’s AI systems as they continue to evolve.

The time cost of creating a well-configured llms.txt is 30 to 60 minutes. The potential benefit — providing AI systems with a clear, curated map of your content — is meaningful enough that this is always worth doing. At worst it has no effect. At best it accelerates your AI indexing and provides AI systems with context that improves the accuracy of their citations from your site.