llms.txt validator: what a valid file needs, and what breaks it

The format is five parts long. Most broken files fail on a link or on a URL that returns HTML instead of Markdown.

·3 min read

An llms.txt validator does two jobs in one pass: it reads your /llms.txt as Markdown to see whether the file follows the published format, and it opens every link in it to see whether an agent could really use that URL. Syntax alone is a low bar. A file with perfect headings still fails if half its links 404.

The AI Agent Readiness Checker is the llms.txt validator at seogeoaeo.ai, and the old /tools/llms-txt-validator address now redirects there. A run costs 10 credits and returns findings with what was observed, why it matters, the fix and a priority order.

What a valid file needs

The format is short. A file at /llms.txt, or at a subpath such as /docs/llms.txt, holds these parts in order:

  • An H1 with the site or project name. This is the only required line.
  • A blockquote summary that says what the thing is and who it is for.
  • Optional context paragraphs carrying facts an agent needs before it opens a link.
  • H2 sections of Markdown links, each [name](https://full-url) with an optional note after a colon.
  • An ## Optional section for pages an agent may skip, such as changelogs and legal pages.
Part Required What a checker tests
H1 name Yes Present, and not HTML
Summary No, but expected Reads as a description, not a slogan
Link sections No Absolute public http(s) URLs
Each link No Loads without a redirect, 404 or login
Optional section No Valid links, secondary pages only

The failures that show up most

  1. The URL returns your homepage. Single-page apps answer unknown paths with a 200 and the app shell, so /llms.txt looks present and is HTML. The first line tells you: # Your Site, not <!doctype html>.
  2. Relative links. [Pricing](/pricing) reads fine to a person and means nothing to a parser that expects a full URL.
  3. A pasted sitemap. Every URL on the site is not a reading list. Pick the pages you would hand a new hire.
  4. Gated or moved pages. A login wall, a redirect chain or a 404 spends the agent's request and returns nothing.
  5. Crawl rules inside the file. Disallow lines and "no training" notes belong in robots.txt. llms.txt does not control access.
  6. A marketing summary. "The leading platform for modern teams" gives a model nothing it can repeat or check.
  7. A CDN challenge. If bot protection answers unknown clients, the agent gets a challenge page instead of Markdown.

How to check one in three minutes

Start with curl: curl -sI https://example.com/llms.txt should show 200 and text/plain or text/markdown, and curl -s should start with your H1. Then open each link by hand and fix the ones that redirect or ask for a login.

For the rest, run the checker on the site URL. It validates the syntax, tests each link as a public URL, and covers the checks Lighthouse runs under Agentic Browsing, Markdown responses for Accept: text/markdown, WebMCP form coverage and MCP or A2A agent cards. You can paste a draft into the optional llmsTxt field to validate it before it ships.

Read the tips from high priority down, apply them, then run again. Tip ids stay the same between runs, so a fix shows up as a missing id and a clean status.

What it will not do

It does not crawl your whole site, keep watching the file after the run, or say whether an assistant will cite you. llms.txt is optional, and Google's own guidance says Search ignores it. Treat a clean run as one less thing an agent can trip over, not as a ranking win. The generator that drafts the file is a separate tool, priced at 15 credits.

FAQ

Is llms.txt a Google ranking factor?

No. Google's guide to AI optimization says Search needs no new AI text files, and that maintaining llms.txt neither helps nor harms visibility there.

Does the file have to sit at the root?

The proposal allows the root or any subpath, and a file covers the pages under its path. /docs/llms.txt is valid for a docs site.

What does a check cost?

10 credits per run for the AI Agent Readiness Checker. Credits come from a monthly plan or a one-time pack, and a run that fails on the platform side is refunded.