iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Short answer: llms.txt has not been shown to improve AI search citations, and Google says Google Search does not use it as a special signal. It may still help an agent or coding tool navigate a documentation site when that tool consults the file. Those are different jobs, so publishing llms.txt can be a practical navigation experiment—but it is not an evidence-backed SEO ranking tactic.
What llms.txt is—and what it is not
llms.txt is a proposed Markdown file, usually placed at a website’s root, that briefly introduces the site and links to selected pages. Its purpose is to orient a model or software agent that has been directed to use the site, particularly when absorbing the entire site would be impractical. The proposal describes an index, not a search-ranking mechanism.
It is not equivalent to robots.txt. Publishing llms.txt does not grant or deny crawling permission, and the proposal does not establish that search engines will automatically fetch the file, follow its links, or cite the site as a result.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Five different steps often get conflated
- Proposal: a format suggests how a site might offer an agent a curated orientation.
- Publication: the site makes a file available at a URL.
- Fetch: a bot or tool requests that URL.
- Use: an agent reads the file and perhaps follows its links.
- Search outcome: a search product retrieves, selects, and cites the site in an answer.
Evidence that a file exists or was fetched does not prove it was used, and evidence of agent use does not by itself prove a search or citation benefit.
What the evidence says about search visibility
Google Search does not use it as a special file
Google’s official AI optimization guide says that site owners do not need new machine-readable files, AI text files, markup, or Markdown to appear in Google Search, including its generative AI features. Google says Search itself does not use llms.txt as a special file. It also says maintaining one for other systems is fine and will neither help nor harm visibility or rankings in Google Search. That statement is about Google Search; it should not be generalized to every agent or software product.
A large observational report found no measurable citation relationship
SE Ranking reported that its analysis of 300,000 domains found no measurable relationship between llms.txt presence and AI citations. This is an observational vendor-reported result, not a controlled test proving that the file can never affect an outcome. It also measures a different question from whether bots request the file.
The reviewed sources do not establish a causal citation uplift from publishing llms.txt. A browser audit recognizing the file, a crawler request, and an AI answer citation are distinct outcomes—not three votes on whether the file “works.”
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsWhat Ahrefs’ server-log study actually measured
Ahrefs analyzed 137,210 domains with traffic in May 2026 using Ahrefs Web Analytics and Bot Analytics. It checked root llms.txt paths, verified that successful responses contained Markdown rather than an error page, and classified requests by user agent. Ahrefs found that 28% of domains in its analytics sample published a file, but cautioned that its customers skew technical and SEO-aware; treat that adoption figure as an upper bound, not a web-wide census.
Among valid files in the sample, 97% received no requests in May 2026. The denominator is valid files identified among sampled domains—not all websites. For files that did receive traffic, 96% of requests came from bots. Of those observed requests, 19.5% came from named AI bot categories, a grouping that combines different purposes such as training, user-directed assistants, agentic infrastructure, and live retrieval.
| Observed request category | Share of requests to files that received traffic | What the figure does—and does not—show |
|---|---|---|
| AI agents and agentic infrastructure | 10.5% | Ahrefs classified this share of observed requests as agentic activity. It is not a share of all agent traffic or proof that a file was used. |
| Live AI retrieval bots | 1.1% | Ahrefs classified this share of observed requests as live retrieval. It is not a share of all AI answers or proof of a citation effect. |
Both percentages describe requests Ahrefs observed to files that got traffic. A server log can show that a URL was requested; it cannot establish that the requester parsed the file, followed its links, completed a task with it, or cited the site in an answer.
Why teams may still ship it for agentic browsing
“Agentic SEO” can mean making a site easier for software agents acting on a user’s behalf to understand and navigate. For a documentation site, a concise index may provide a useful starting point when a coding or browsing agent has been directed to find relevant material. That narrower possibility fits the request pattern better than a claim about live AI search citations: Ahrefs observed more requests from agentic infrastructure than from live retrieval bots.
Chrome’s Lighthouse documentation includes an experimental llms.txt audit in an Agentic Browsing category. That signals browser-tooling interest in whether a site supplies an agent-navigation file; it does not mean Google Search consumes it or establish a measured citation benefit.
Best Value
The proposal author, Jeremy Howard, has described the expected use as “mainly useful for inference, i.e. at the time a user is seeking assistance.” That is a rationale for possible agent use, not evidence that a particular search product reads the file.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Should your site publish llms.txt?
| Site or goal | Practical decision | Measure success by |
|---|---|---|
| Developer documentation with a clear need for agent navigation | Consider a short, maintained index pointing to the most useful, controlled destinations. | Whether relevant agents follow its links and complete documentation tasks—not file existence alone. |
| General website seeking Google Search visibility | Do not prioritize llms.txt as a ranking or citation tactic. Follow Google’s published Search guidance and ensure content is crawlable and eligible. | Google Search visibility, using Search Console and other appropriate reporting—not an llms.txt audit score. |
| Site already publishing a file | Keep it accurate and limited to destinations that help the intended agent; remove stale links. | Requests in server logs and, where observable, whether relevant agents follow links and accomplish the intended task. |
For a low-cost documentation experiment, write the file for a real navigation problem rather than for an assumed search boost. Check your own server logs, but interpret a fetch as a fetch. A one-off request, a passed audit, or the presence of the file is not proof of a visibility gain.
Why “half the data says it doesn’t work” is misleading
The headline framing is not a literal count of comparable studies split between success and failure. The surfaced evidence measures different things: file adoption, request frequency, a reported correlation with citations, and browser-audit readiness. None is interchangeable with a controlled test of whether llms.txt changes AI answer citations. The most defensible conclusion is narrower: Google Search says it ignores the file, the available observational evidence does not demonstrate citation gains, and a navigation benefit for some agents remains plausible but not established as a general result.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
