How to Add llms.txt to Your Website (Step-by-Step)

Key Takeaways

An llms.txt file helps AI crawlers understand your site’s content structure — think of it as a robots.txt for large language models.

You can create one manually or generate it automatically with a tool like LLMGenerator.

Announcing it via the <link rel="llms-txt"> HTML tag lets AI systems discover it without crawling your full site.

The whole process takes under 15 minutes on most platforms.

AI systems are increasingly being used to research, summarize, and cite web content. According to a 2025 BrightEdge study, AI-powered search features influenced 58% of informational queries (BrightEdge, 2025). Getting your content structured for AI discovery is no longer optional — it’s a competitive advantage. That’s exactly what an llms.txt file does, and this guide shows you how to add one to your site today.

What Is llms.txt?

The llms.txt standard (defined at llmstxt.org) is a plain-text Markdown file you place at the root of your domain — yourdomain.com/llms.txt. It gives AI language models a curated, structured overview of your site’s most important content.

Where robots.txt tells crawlers what not to index, and sitemap.xml lists every URL, llms.txt tells AI systems what your content is about and which pages matter most. It’s written in clean Markdown, which LLMs parse naturally.

A minimal llms.txt looks like this:

# Your Website Name

> A one-sentence description of what your site offers.

## Docs

- [Getting Started](/docs/getting-started): How to set up the product.
- [API Reference](/docs/api): Full API documentation.

## Blog

- [Latest Post Title](/blog/post-slug): A short description.

Step 1: Create Your llms.txt File

You have two options:

Option A — Generate it automatically

Use LLMGenerator to scan your site and produce a properly formatted llms.txt in seconds. This is the fastest path and handles edge cases like pagination, canonical URLs, and dynamic content.

Option B — Write it manually

Create a file named llms.txt and structure it with Markdown headings and links. Keep it focused: include your most important pages, not every URL on your site. AI systems benefit from curation, not exhaustive lists.

Tips for a great llms.txt:

Write a clear, one-sentence > blockquote description at the top.
Group links under meaningful ## section headings.
Add a short description after each link’s colon.
Keep the total file under 100KB — ideally under 20KB.

Step 2: Upload It to Your Site Root

Your llms.txt must be accessible at https://yourdomain.com/llms.txt. How you upload it depends on your platform:

Platform	How to upload
Static site (Astro, Next.js, etc.)	Place in the `/public` directory
WordPress	Upload via FTP to the root or use a plugin
Webflow	Use the Asset Manager or a custom domain redirect
Shopify	Add via Theme Editor > `assets/` folder with a redirect
Wix	Upload via Wix Dev Mode or use a redirect rule
Squarespace	Use Code Injection or a third-party file host with a redirect

After uploading, verify it’s live by visiting https://yourdomain.com/llms.txt directly in your browser.

Step 3: Announce It With the HTML Link Tag

This is the step most tutorials skip. Once your file is live, you should announce it in your HTML <head> so AI crawlers and browsers can discover it automatically — without having to guess the URL.

Add this tag inside the <head> of every page (or at minimum your homepage):

<link rel="llms-txt" href="/llms.txt" />

This follows the same convention as other resource hints (<link rel="sitemap">, <link rel="manifest">). It’s a declarative signal: “this site has an llms.txt file, and here’s where to find it.”

The rel="llms-txt" attribute isn’t yet a W3C standard, but it’s gaining adoption among early implementers and is referenced in community tooling. Publishing it now positions your site ahead of the curve before formal standardization happens.

Where to add it in common frameworks

Astro (src/layouts/Layout.astro or BaseHead.astro):

<link rel="llms-txt" href="/llms.txt" />

Next.js (app/layout.tsx):

export default function RootLayout({ children }) {
  return (
    <html>
      <head>
        <link rel="llms-txt" href="/llms.txt" />
      </head>
      <body>{children}</body>
    </html>
  );
}

WordPress (functions.php):

function add_llms_txt_link() {
  echo '<link rel="llms-txt" href="/llms.txt" />' . "\n";
}
add_action('wp_head', 'add_llms_txt_link');

Step 4: Verify Everything Works

After adding the file and the link tag, do a quick sanity check:

File accessible: Visit https://yourdomain.com/llms.txt — you should see your Markdown content.
Link tag present: Open DevTools > Elements and search for llms-txt in the <head>.
No redirect issues: The URL should return HTTP 200, not a redirect chain.
File is valid Markdown: Paste the content into any Markdown previewer and confirm it renders cleanly.

Does llms.txt Actually Help?

Worth asking. No major LLM provider has officially confirmed they parse llms.txt files in their training pipelines. OpenAI, Google, and Anthropic all respect robots.txt — but llms.txt adoption is still emerging.

That said, there are real, practical benefits right now:

MCP tooling — Model Context Protocol (MCP) servers can use your llms-full.txt as a knowledge source directly (more on this in our MCP guide).
Developer discovery — Developers building AI agents often point those agents at llms.txt files to ground them in your docs.
Future-proofing — As the standard matures, sites with well-maintained llms.txt files will have a head start.

FAQ

Does adding llms.txt affect my regular SEO? No. It’s a separate file that doesn’t interact with Googlebot or traditional search crawlers. Your robots.txt and sitemap.xml remain unchanged.

How often should I update llms.txt? Update it whenever you add major new sections, change your site structure, or publish high-value content you want AI systems to discover. Monthly reviews are a good cadence.

Should I also create an llms-full.txt? Yes, if you have substantial documentation or long-form content. The llms-full.txt file contains the full text of your key pages — useful for MCP integrations and AI agents that need deep context.

What if my site has multiple subdomains? Each subdomain should have its own llms.txt at its root. You can cross-reference them using the Optional section in each file.

Adding llms.txt is a low-effort, high-signal move for any site that cares about how AI systems represent its content. Create the file, upload it to your root, add the <link rel="llms-txt"> tag to your <head>, and you’re done. Fifteen minutes of work for a meaningful advantage as AI-driven discovery keeps growing.

Ready to generate your llms.txt automatically? Try LLMGenerator for free.