An llms.txt file is a Markdown-formatted text file placed at your website's root directory that tells AI language models (ChatGPT, Claude, Perplexity, Gemini) which pages matter most and how to read them. Creating one takes under 15 minutes: write a structured Markdown document with your site title, a brief description, and H2-organized links to your key pages, then upload it to `yourdomain.com/llms.txt`. It is one of the fastest, zero-cost steps B2B marketers can take to improve AI discoverability right now.

What Is an llms.txt File and Why Does It Matter for AI Search?

An llms.txt file is a plain-text, Markdown-formatted document placed at the root of your website that gives AI language models a curated, easy-to-parse map of your most important content. It cuts through the ads, navigation menus, and JavaScript that make standard HTML pages hard for AI to process.

The official specification at llmstxt.org is direct about why the file exists: "websites are not optimized for LLM consumption." Standard HTML pages carry menus, cookie banners, ad scripts, and JavaScript rendering that degrade AI comprehension.

Zeo Agency's analysis describes llms.txt as a Markdown file in the root directory that helps AI engines including ChatGPT, Gemini, Claude, and Perplexity process site content accurately by summarizing the essentials in a format these models already handle well.

The problems llms.txt addresses directly:

• JavaScript rendering that blocks AI crawlers from reading dynamic content
• Ad and cookie banner clutter that dilutes page signal
• Navigation bloat that buries the content AI needs to cite
• Irrelevant boilerplate (footers, legal disclaimers, sidebar widgets) that consumes context window space

According to Sona AI Visibility, 3 in 4 websites are partially or fully invisible to AI engines. The platform checks for a valid llms.txt file as one of 17 audit checks across Crawlability, Schema Markup, Content Structure, and Freshness categories. The absence of an llms.txt file is one of the fastest issues to fix, and it costs nothing.

Who Created the llms.txt Standard and Is It Official?

The llms.txt standard was proposed by Jeremy Howard in September 2024 and is documented at llmstxt.org. It is not a formal W3C or IETF standard, but it has seen rapid organic adoption across documentation platforms, SaaS companies, and developer tools.

The official proposal specifies the Markdown format, placement convention, and file structure: an H1 for the site or project name, an optional blockquote summary, non-heading sections for additional detail, and H2 headers delimiting organized lists of links with optional plain-text descriptions. The spec is parseable via regex or any standard Markdown parser, which is part of why adoption has moved quickly among developer-facing teams.

GitBook's documentation notes that the standard is "still evolving" and not yet universally parsed by all major LLMs. Creating the file now is a forward-looking investment, not a guaranteed immediate ranking lever.

Mintlify's assessment is worth reading carefully. The llms-full.txt variant (which contains full plain-text site content rather than a curated index) exists alongside the standard llms.txt, and the gap between the proposal and confirmed LLM parsing behavior is still real. Perplexity has shown the most openness to structured AI-guidance files. ChatGPT's GPTBot respects robots.txt but llms.txt parsing is not yet officially confirmed. Google AI Overviews relies primarily on its existing indexing infrastructure.

Current support status by engine:

• Perplexity: Most receptive to structured AI-guidance files
• Claude (Anthropic): No confirmed native parsing of llms.txt at root
• ChatGPT (OpenAI): GPTBot respects robots.txt; llms.txt parsing unconfirmed
• Google AI Overviews: Relies on existing indexing; experimental at best

How Do You Create an llms.txt File Step by Step?

Creating an llms.txt file requires no special tools. Open any text editor, follow the Markdown structure from the official spec, list your most important pages with descriptions, save the file as `llms.txt`, and upload it to your website's root directory.

Hostinger's tutorial outlines the three core steps: write Markdown with URLs, titles, and content summaries; save as llms.txt; upload to the root directory. GitBook recommends starting with an H1 title, grouped section headings, links to key pages, and short descriptions for each link. Firecrawl's walkthrough covers crawl-based generation for sites where manual curation would take too long.

Step-by-step process:

1. Open a plain text editor (VS Code, Notepad, TextEdit in plain text mode)
2. Write your H1 site title on line one
3. Add a blockquote summary on line two (one sentence describing what your site does)
4. Add H2 section headers to organize your content categories
5. Under each H2, list your key pages as hyphenated Markdown links with short descriptions
6. Add an `## Optional` section at the bottom for lower-priority pages
7. Save the file as `llms.txt` (not `llms.md`, not `llms.txt.md`)
8. Upload to your root directory so it resolves at `yourdomain.com/llms.txt`
9. Verify by visiting that URL in a browser

A minimal valid llms.txt file:

```

Acme Corp

B2B SaaS platform for revenue operations teams.

Product Docs

• Getting Started: Onboarding guide for new users.
• API Reference: Full REST API documentation.

Blog

• How We Reduced Churn by 40%: Case study with methodology.

Optional

• Privacy Policy

```

That is a complete, valid file. Most sites can produce one in under 15 minutes.

What Are the Formatting Rules and Best Practices for llms.txt?

The llms.txt specification requires valid Markdown with a single H1 title, an optional blockquote summary, H2 section headers organizing groups of links, and plain-text descriptions. No XML, no HTML, no JavaScript.

The official llmstxt.org specification is precise: H1 for the site or project name, blockquote for the summary, H2 headers for file list sections, and hyphenated links with optional descriptions after a colon. The file must be parseable by a standard Markdown parser or a simple regex. That means no custom syntax, no embedded scripts, and no HTML tags.

David Dias's practitioner guide on Medium covers common formatting mistakes: using H3 headers instead of H2 for sections (which breaks the spec), including HTML anchor tags instead of Markdown links, and omitting descriptions entirely (which reduces value to AI engines parsing the file). Webflow University's demonstration shows how to handle CMS-driven sites where content is dynamic, recommending a manually maintained llms.txt updated when major content changes occur.

Formatting rules at a glance:

ElementRequired?FormatExampleSite titleYes`# Title``# Acme Corp`SummaryRecommended`> blockquote``> CRM for B2B SaaS teams`Section headersYes (if multiple sections)`## Section Name``## Docs`Page linksYes`- Title: Description``- API Docs: REST reference`DescriptionsRecommendedPlain text after colon`Onboarding guide for new users`Optional sectionOptional`## Optional`Low-priority pagesFile formatMandatory`.txt`, plain Markdown`llms.txt`PlacementMandatoryRoot directory`yourdomain.com/llms.txt`

Best practices:

• Keep descriptions under 20 words per link. AI engines parse these as context signals, not marketing copy.
• Prioritize your highest-value pages in the first two sections. AI models read top-to-bottom and context windows are finite.
• Use an `## Optional` section for lower-priority content so AI engines know what to deprioritize.
• Curate, do not dump. Listing every page defeats the purpose. Aim for 10 to 30 links for most B2B sites.
• Update the file when major content changes occur, particularly when new product pages, case studies, or documentation sections launch.

Where Should You Place the llms.txt File on Your Website?

Your llms.txt file must be placed at the root of your domain, accessible at `https://yourdomain.com/llms.txt`, following the same convention as robots.txt and sitemap.xml so AI crawlers can discover it without additional configuration.

Zeo Agency confirms the file must be in the website root directory for AI discovery by ChatGPT, Claude, and Perplexity. Hostinger explains that root placement is what allows AI crawlers to find the file without being directed to it through any other mechanism. A Webflow University video walkthrough demonstrates the CMS-specific upload process for Webflow sites in detail.

Platform-specific placement guide:

• WordPress: Upload via FTP/SFTP to `/public_html/llms.txt` or use a file manager plugin from your hosting control panel
• Webflow: Upload as a static asset via the Assets panel, then configure the path to resolve at `/llms.txt`
• Mintlify: Built-in llms.txt generation is auto-placed at root; no manual upload required
• GitBook: Platform-native support auto-generates and places the file from your existing doc hierarchy
• Next.js / custom builds: Add `llms.txt` to your `/public` directory; it will resolve at root automatically
• Shopify: Upload via theme files editor or a custom app that serves static files at root

If your documentation lives at `docs.yourdomain.com`, place a separate llms.txt at `docs.yourdomain.com/llms.txt` in addition to your main domain file. AI engines treat subdomains as distinct properties.

Are There Tools to Generate an llms.txt File Automatically?

Several free and platform-native tools can generate an llms.txt file automatically by crawling your sitemap or exporting your documentation structure, removing the need for manual page-by-page curation on large sites.

Mintlify offers automatic llms.txt generation for documentation sites, producing both `llms.txt` and `llms-full.txt` variants from your existing doc structure with no manual input required. GitBook has built-in support that auto-generates the file from your existing documentation hierarchy. For sites not on either platform, the LLMrefs free online generator creates a valid file from a URL input with no account required. Firecrawl's crawl-based approach works for larger or custom-built sites by scraping your sitemap and extracting page titles and descriptions programmatically via API.

Tool comparison:

ToolTypeBest ForCostllms-full.txt?Auto-updates?MintlifyPlatform-nativeDeveloper docsFree tier availableYesYesGitBookPlatform-nativeProduct/SaaS docsFree tier availablePartialYesLLMrefs GeneratorStandalone toolAny websiteFreeNoNoFirecrawlAPI/crawl-basedLarge sites, custom buildsFreemiumYesVia APIManual (text editor)DIYSmall sites, full controlFreeOptionalManual

For sites with fewer than 50 pages, manual creation gives the most control and takes under 15 minutes. For documentation-heavy products with hundreds of pages, automated generation via Mintlify, GitBook, or Firecrawl is the practical choice. The tradeoff is curation: automated tools list everything, while manual files let you prioritize ruthlessly.

How Does llms.txt Compare to robots.txt and Sitemaps, and Does It Actually Help AI Visibility?

llms.txt, robots.txt, and sitemaps serve different purposes. robots.txt controls crawler access permissions. Sitemaps list URLs for indexing. llms.txt gives AI engines curated, human-readable context about what your content means. It is the only one of the three designed for AI comprehension rather than crawl mechanics.

Hostinger draws the direct comparison: robots.txt handles crawlability, sitemaps handle URL discovery, and llms.txt provides content summaries for better AI understanding. None of the three replaces the others. Zeo Agency frames llms.txt as a Generative Engine Optimization (GEO) tool, distinct from traditional SEO signals that target Google's ranking algorithm.

Mintlify's balanced assessment is worth taking seriously: adoption is still early, confirmed parsing by major LLMs is not yet universal, and llms.txt should be treated as a forward-looking investment. Setup takes under 15 minutes and costs nothing. The risk-reward calculation is straightforward.

Three-file comparison:

Featurerobots.txtsitemap.xmlllms.txtPrimary purposeControl crawler accessList URLs for indexingGuide AI content comprehensionFormatPlain text directivesXMLMarkdownTarget audienceAll crawlersSearch engine indexersAI language modelsContent includedAllow/disallow rulesURLs + metadataTitles, descriptions, contextPlacementRoot (`/robots.txt`)Root or subdirectoryRoot (`/llms.txt`)Required for SEO?YesRecommendedNo (but emerging)Required for AEO?PartialPartialRecommendedSupported by Google?YesYesExperimentalSupported by ChatGPT?Partial (GPTBot)NoEmerging

60% of Google searches now end without a click, making AI-generated answers an increasingly important traffic channel for B2B content. Teams running audits with Sona AI Visibility can check for llms.txt presence and validity as part of the platform's Crawlability category, alongside a live GPTBot probe and robots.txt validation. The free scan takes under 30 seconds and surfaces exactly which files are missing or misconfigured.

Frequently Asked Questions

Can you show me how to create an llms.txt file for my website?

Open a text editor, start with `# Your Site Name` on line one, add a `> one-sentence description` blockquote on line two, then add `## Section` headers with `- Page Title: Short description` lines for each key page. Save the file as `llms.txt` and upload it to your root directory so it resolves at `yourdomain.com/llms.txt`. The process takes under 15 minutes for most sites. Verify it worked by visiting that URL in a browser and confirming the file loads as plain text.

What should I include in an llms.txt file to help AI language models?

Include your site name as an H1, a one-sentence summary as a blockquote, and organized sections of links to your most important pages with short plain-English descriptions. Prioritize product pages, documentation, case studies, and high-value blog content. Avoid listing every page: curation is the point. An `## Optional` section at the bottom can hold lower-priority URLs without cluttering the primary content AI engines read first.

Is there a recommended format or structure for an llms.txt file?

Yes. The official specification at llmstxt.org defines the format: valid Markdown only, H1 for the site title, an optional blockquote summary, H2 headers to organize sections, and hyphenated links with plain-text descriptions after a colon. No XML, no HTML tags, no JavaScript. Using H3 headers for sections or embedding HTML anchor tags breaks compatibility with tools and crawlers that validate against the spec.

How do I generate an llms.txt file automatically from my docs?

If you use Mintlify or GitBook, both platforms offer built-in llms.txt generation from your existing documentation structure with no manual effort. For other sites, the LLMrefs free online generator creates a valid file from a URL input, and Firecrawl provides a crawl-based API approach for larger or custom-built sites. Automated tools work best for documentation-heavy products; smaller sites with fewer than 50 pages are faster to handle manually.

Why is it important to have an llms.txt file on my site?

AI engines like ChatGPT, Perplexity, and Google AI Overviews struggle to parse standard HTML pages cluttered with navigation, ads, and JavaScript. An llms.txt file gives these engines a clean, structured summary of your most important content, increasing the likelihood your site is cited in AI-generated answers. With 60% of Google searches now ending without a click, AI citation is becoming a primary discovery channel for B2B brands, and the file costs nothing to create.

What is the difference between llms.txt and llms-full.txt?

`llms.txt` is a curated index: a short Markdown file linking to your key pages with brief descriptions, designed for quick AI orientation. `llms-full.txt` is a longer variant containing the full plain-text content of your site or documentation, allowing AI models to read everything in one file without following individual links. Mintlify and Firecrawl support both variants. For most B2B SaaS sites, start with `llms.txt` and add `llms-full.txt` only if your documentation is the primary asset you want AI engines to consume in full.

Does llms.txt work with ChatGPT, Perplexity, and Google AI Overviews?

Adoption is still emerging as of April 2026. The standard was proposed in September 2024 and is gaining traction, but confirmed parsing behavior varies by engine. Perplexity has shown the most openness to structured AI-guidance files. ChatGPT's GPTBot respects robots.txt directives but llms.txt parsing is not yet officially confirmed. Google AI Overviews relies primarily on its existing indexing infrastructure. Creating the file now positions your site ahead of the curve as support formalizes across engines.

Do I need an llms.txt file if I already have a sitemap and robots.txt?

Yes, because they serve different purposes. robots.txt tells crawlers what they can access. Sitemaps list your URLs for indexing. llms.txt tells AI engines what your content means and which pages are worth reading. It does not compete with robots.txt or sitemaps. It adds the comprehension layer that neither of those files provides.

Last updated: April 2026