Cloudflare CEO Matthew Prince criticized Google’s approach to AI crawling after Cloudflare reported blocking 416 billion AI bot requests for its customers since July 1, 2025. The figure is a company-reported count of requests—not a verified tally of unique bots or AI-training events. Prince’s central concern is that publishers may find it difficult to restrict AI access without also affecting Google Search indexing.
What does Cloudflare’s 416 billion figure mean?
Cloudflare told WIRED that it had blocked 416 billion AI bot requests for customers from July 1, 2025, through Prince’s December 2025 interview at WIRED’s Big Interview event in San Francisco. That is Cloudflare’s reported request count; WIRED did not describe it as independently audited. A request is not necessarily a unique bot, a unique page, or a completed training event. WIRED’s December 4, 2025 report is the source of the figure.
The number illustrates the scale of crawler traffic Cloudflare says it handled across its customers. It does not establish how many individual AI systems were blocked, what content they would otherwise have accessed, or whether any particular publisher lost or gained traffic as a result.
Why did Matthew Prince criticize Google?
Prince’s criticism was about the practical choices available to publishers. As WIRED reported his remarks, Google had combined search and AI crawling. A publisher that blocks the AI crawler could therefore risk blocking the crawler used to index pages for Google Search. That can put two goals in tension: limiting AI systems’ access to content while preserving visibility in a search engine that may send readers to the site.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitches#1 Best Overall
- Attention-grabbing design meets the latest evolution of the Google Pixel Camera on the new Google Pixel 11 Pro; Gemini Intelligence helps manage details so you can live in the moment[1]; and the phone is available in two sizes
- Unlocked Android phone gives you the flexibility to change carriers and choose your own data plan: Works with Google Fi, Verizon, T-Mobile, AT&T, and other major carriers[2]
- Stay informed without looking at your screen: When your phone is face down, Pixel HiLight gently alerts you with subtle glowing lights when your favorite contacts are calling or you’re talking with Gemini; exclusive to Google Pixel 11 Pro phones
- Magic Capture catches the moment as you live it: With just one tap, Pixel 11 Pro captures video and photos, and automatically edits, crops, and unblurs a curated collection, ready to share – and you get the memory of how it felt to be in the moment
- Two new cameras for more brilliant photos: A larger telephoto sensor captures 30% more light for clear, beautiful photos and videos, even in the dark[3]; Pixel’s longest zoom ever helps you capture details from impressive distances[4]
Prince called for Google to separate its search and AI crawlers, arguing that publishers should be able to set different access rules for each. His charge that Google was using its existing position to gain leverage in AI is his criticism, not an independently established finding about Google’s intent. The consequences of blocking search indexing will also vary by publisher, audience, and reliance on Google referrals; the reported arrangement does not mean every site faces the same business outcome.
In July 2025, as Cloudflare introduced controls for blocking AI crawlers or requiring payment for scraping, publishers and SEO practitioners raised the issue of how to block Google’s AI use without blocking search indexing. Ars Technica reported that Google declined to confirm whether discussions with Cloudflare were underway or whether it was willing to separate its crawlers. That report does not establish a later agreement or a confirmed Google response to Prince’s December criticism. Ars Technica’s July 2025 coverage describes that earlier dispute.
Rank #2
- Google Pixel 10a is a durable, everyday phone with more[1]; snap brilliant photography on a simple, powerful camera, get 30+ hours out of a full charge[2], and do more with helpful AI like Gemini[3]
- Unlocked Android phone gives you the flexibility to change carriers and choose your own data plan; it works with Google Fi, Verizon, T-Mobile, AT&T, and other major carriers
- Pixel 10a is sleek and durable, with a super smooth finish, scratch-resistant Corning Gorilla Glass 7i display, and IP68 water and dust protection[4]
- The Actua display with 3,000-nit peak brightness shows up clear as day, even in direct sunlight[5]
- Plan, create, and get more done with help from Gemini, your built-in AI assistant[3]; have it screen spam calls while you focus[6]; chat with Gemini to brainstorm your meal plan[7], or bring your ideas to life with Nano Banana[8]
What do the crawler and referral figures show?
Prince cited Cloudflare analysis claiming Google saw 3.2 times as many pages as OpenAI, 4.6 times as many as Microsoft, and 4.8 times as many as Anthropic or Meta. These comparisons were relayed by Prince to WIRED; the interview report did not provide the underlying methodology. They should be read as figures attributed to Cloudflare analysis, not as independently verified measurements of all web crawling.
A separate Cloudflare analysis published in June 2025 compared HTML crawl requests associated with platforms against HTML requests whose Referer header named those platforms. It reported the following crawl-to-referral ratios:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
| Platform | Cloudflare-reported ratio | How to interpret it |
|---|---|---|
| OpenAI | 1,700 crawl requests per referral | Cloudflare’s June 2025 calculation using observed platform-associated HTML requests and HTML requests with a matching Referer header. |
| Anthropic | 73,000 crawl requests per referral | Cloudflare’s June 2025 calculation using the same method. |
Cloudflare cautioned that referrals from native apps may not include a Referer header, so its method can miss some visits. The ratios are therefore a measure of what that particular method observed in Cloudflare’s data, not a complete count of traffic each platform sent to publishers. Cloudflare’s analysis and product announcement explains the calculation and its caveat.
What can publishers do to manage AI crawlers?
Cloudflare described managed robots.txt and controls for blocking AI bots, including the ability to apply restrictions selectively to parts of a site monetized through advertising. It said more than one million customers had enabled its one-click AI scraper-blocking feature as of its 2025 post; that is a company-reported adoption figure, not an independent survey.
Rank #4
- Google Pixel 10 Pro is the ultimate Pixel experience, featuring advanced AI with Gemini, unbelievable camera quality, impeccable design in two sizes, and the next-gen Google Tensor G5 chip[1]
- Unlocked Android phone gives you the flexibility to change carriers and choose your own data plan[2]; it works - Google Fi, Verizon, T-Mobile, AT&T, and other major carriers
- Get a head start on syncing your data before it even arrives: After you purchase your new Pixel, look for an email that explains how to transfer your photos, videos, passwords, and more in just a few quick steps[11]
- Pixel’s pro camera system makes everything look amazing, even in low light; capture more of the scene with advanced Google AI models, and bring out incredible details with 100x Pro Res Zoom, stunning 50 MP images, and super steady videos in 8K[10]
- Pixel 10 Pro is built with durable aluminum and Corning Gorilla Glass Victus 2 for scratch and drop resistance; the 6.3-inch Super Actua display with 3,300-nit peak brightness is easy on the eyes, even in direct sunlight[3,13,18]
These controls address how a site communicates or enforces access preferences. They do not, by themselves, resolve the search-versus-AI problem if the same crawler serves both functions. Publishers assessing an approach should distinguish among the rule they want to set, the crawler or purpose the rule can identify, and the commercial or discovery effects that may follow.
- Block access: A site can use available controls to deny identified AI crawlers access, where the crawler can be distinguished and the control applies. Blocking does not create a licensing arrangement or guarantee that every request will be identified.
- Set preferences: A managed robots.txt can communicate rules to crawlers that follow those instructions. It is a way to express access preferences, not a mechanism that proves every crawler will comply.
- Allow or license access: Cloudflare’s September 2025 announcement presented permission-based access and a possible new economic model. Publisher endorsements quoted in that announcement are stakeholder views; they do not demonstrate that a broad, established licensing market exists. Cloudflare’s September 2025 announcement describes that direction.
The useful questions are whether search indexing remains available, whether AI training or answer-generation access is restricted, how reliably crawler identity and purpose can be distinguished, and what traffic measurement can actually observe. A setting that controls access may still leave a publisher with a tradeoff when search and AI crawling are not separated.
Best Value
- Google Pixel 7 is powered by Google Tensor G2; it’s faster, more efficient, and more secure, with the best photo and video quality yet on Pixel[1].Other camera description:Front,Rear.Bluetooth Version 5.2 with dual antennas for enhanced quality and connection.
- Unlocked Android 5G phone gives you the flexibility to change carriers and choose your own data plan[2]; works with Google Fi, Verizon, T-Mobile, AT&T, and other major carriers
- Pixel’s Adaptive Battery can last over 24 hours; when Extreme Battery Saver is turned on, it can last up to 72 hours[3]
- The 6.3-inch Pixel 7 display is super sharp, with rich, vivid colors; it’s fast and responsive for smoother gaming, scrolling, and moving between apps[4]
- Google Pixel 7 has wide and ultrawide lenses with up to 8x Super Res Zoom[5]; and Cinematic Blur brings more drama to your videos
What publishers and readers should take away
Cloudflare’s 416 billion figure points to the volume of AI bot requests the company says it blocked for customers over a period beginning July 1, 2025. Prince used that figure to argue for more meaningful publisher control, focusing particular criticism on Google’s reported combination of search and AI crawling. The underlying issue is not just whether a site can block a bot: it is whether publishers can apply different rules to different uses without sacrificing a route by which readers find their work.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
