Surfaced AI

What is the difference between llms.txt and robots.txt?

robots.txt controls what crawlers may access; llms.txt describes what your site is. One is permission, the other is context.

robots.txt is an access-control convention. It tells crawlers which paths they may fetch, and it is widely respected by the major agents.

llms.txt is descriptive rather than restrictive. It offers a short, curated summary of your site and its key pages, so a system reading it gets your positioning directly instead of inferring it.

They are not alternatives and do not interact. Blocking a crawler in robots.txt is not undone by describing yourself in llms.txt, and vice versa.

Practically: use robots.txt to decide who may read you, and llms.txt to shape what they conclude once they do.

See where you actually stand

32 checks, four pillars, one score — free and no account needed.

Run free audit

More in this category

What is llms.txt?A plain-text file at your site root that summarises what your site is and points to the pages worth reading.Do I actually need an llms.txt file?It is optional and not universally read, but it is cheap enough that there is little reason to skip it.What should I put in my llms.txt?Your name, a one-line summary, a short description, and a curated list of your most useful pages.
All answers

Your competitors are already being recommended. Catch up.

Run the audit. Decide after. Understand why AI engines recommend competitors instead of you. Free, in about two minutes. No card, no signup, no obligation.

Or browse the sample audit first