iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Cloudflare Markdown for Agents has not been shown to improve AI visibility. In a February 2026 randomized experiment, pages served as Markdown did not receive a statistically significant increase in visits from AI crawlers compared with pages served as HTML. That is evidence of no demonstrated impact in that experiment—not proof that Markdown can never affect crawler activity, citations, or referrals.
What the February 2026 experiment found
Brandon Punturo’s experiment randomized pages from six websites to HTML or Markdown treatments and tracked known AI crawler visits from January 19 to February 8, 2026. The detailed sample reports 380 pages in the analysis after one unusually high-traffic page was excluded; the paper’s abstract initially describes 381 assigned pages.
The Markdown treatment served clean Markdown to detected LLM bot user agents, while human visitors and non-LLM crawlers received standard HTML. The control pages served HTML. The measured outcome was unique bot requests per tracked page, deduplicated at one-minute granularity.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
| Measure | HTML control | Markdown treatment | How to read it |
|---|---|---|---|
| Median bot visits per page | 6.0 | 7.0 | Descriptive medians; the overall result was not statistically significant. (Punturo, 2026) |
| CUPED-adjusted mean lift | +13.9% | 95% confidence interval: −11.2% to +46.8%; it includes zero, so the experiment did not establish a lift. (Punturo, 2026) | |
The positive point estimate is compatible with the data, but so are a modest decrease and a larger increase. Punturo’s paper says the design had an approximately 41% minimum detectable effect and could not confidently rule out moderate effects in the 10–15% range. The correct conclusion is therefore that the experiment did not detect a statistically significant overall crawler-visit increase—not that Markdown has been proved to have exactly zero effect.
#1 Best Overall
Which bots and outcomes were counted
The paper lists OpenAI’s ChatGPT-User, OAI-SearchBot and GPTBot; Anthropic’s ClaudeBot; PerplexityBot; Meta/Facebook bots; and DuckDuckGo DuckAssistBot. Googlebot and Bingbot were excluded because the authors could not reliably separate AI-related activity from ordinary search indexing.
The study measured crawler requests to pages. It did not measure whether a page was cited in an AI-generated answer, how prominently it appeared there, its search or answer ranking, or referral traffic. Punturo explicitly distinguishes crawl volume from citations, and notes that crawl activity and downstream visibility are not perfectly correlated.
What Cloudflare Markdown for Agents does
Cloudflare describes Markdown for Agents as an on-the-fly conversion of HTML to Markdown for enabled zones when a requester negotiates the text/markdown media type. It gives a requesting client a different representation of a page, with human-oriented presentation removed. The feature documentation describes content delivery; it does not establish Markdown as a ranking or citation signal. Cloudflare’s Markdown for Agents documentation
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchThat distinction matters: a crawler must request and receive the Markdown representation for the format change to apply. The existence of a Markdown response does not itself show that a bot will visit more often or that an AI system will select the page for an answer.
What this means for citations and traffic
“AI visibility” can refer to several different outcomes, and they should not be treated as interchangeable:
- Crawler visits: bots request pages. This is what the experiment measured.
- Answer inclusion: a page is cited or used in a generated response. The experiment did not test this.
- Citation prominence: a page appears more visibly or higher in an answer. This was not measured.
- Referral traffic: people click through from an AI product to the site. This was not measured.
It is possible for crawl behavior and citations to move differently. The experiment’s authors call for direct measurement of citation rates rather than assuming that more bot requests mean more citations. No result in this study supports promising that enabling Markdown for Agents will increase any of these downstream outcomes.
How to judge an AI-visibility claim about Markdown
A useful evaluation separates a change in response format from a change in access and from a change in outcomes. If you test Markdown delivery on your own site, look for evidence that isolates those variables:
- Measure the outcome you care about. Track crawler visits separately from citations, citation prominence, rankings and referral sessions.
- Verify crawler access. Check that relevant crawlers can fetch the pages and are not blocked or throttled by bot-management settings or rate limits.
- Keep page meaning equivalent. Compare the Markdown and HTML versions for complete content and preserved structure, rather than assuming conversion alone is sufficient.
- Use a credible comparison. Randomize treatment and control pages, define which bots count before the test, include enough pages and sites, and document how outliers are handled.
- Read uncertainty and duration. A short test or wide confidence interval may not rule out meaningful effects; look for replication across sites, content types and AI systems.
Clear semantic structure, crawlable pages, fast loading and avoiding bot-hostile access barriers are reasonable practices suggested by the experiment’s authors, but they were not separate treatment arms in this test. The study therefore does not establish that any one of those practices—or Markdown formatting itself—caused better AI visibility.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Cloudflare crawler controls are a separate issue
Cloudflare provides separate controls for Search, Agent and Training behavior. Access policies can change whether crawlers reach a site, which in turn can affect observed visits; those controls should not be confused with the format of a response. Cloudflare describes these categories in its September 30, 2026 overview of the internet’s second audience and documents AI bot blocking in Block AI Bots. Product defaults and labels can change, so consult the current documentation for operational settings.
Cloudflare also reported on July 1, 2026 that more than 50% of AI crawler crawl traffic on its network was spent re-fetching unchanged pages. That is Cloudflare’s own network estimate about crawl efficiency, not evidence that Markdown increases visibility. Its September 2026 blog separately reported daily AI-agent requests on its network had grown by more than 1,700% over the previous year; that broad company-reported figure likewise says nothing about Markdown’s effect. Cloudflare’s July 1, 2026 announcement
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
