Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
In one shopping-agent run dated September 19, 2026, Christian Anderson classified 56 of 85 fashion sources as OK, 22 as blocked, four as parse failures, one as reachable but yielding nothing parsed, and two as errors. Only 45 sources returned any items. The main practical finding: structured product data made price and variant checks easier at some stores, but a product price did not prove that the item was available in the shopper’s size. These are the author’s results from one personal project run, not an independently replicated benchmark or a claim about all retailers.
What the 85-source run measured
Anderson’s report, published September 30, 2026, describes a single daily source-gather run by a personal shopping agent. Its 85-source count is the author’s selected source list and classifications for that run. The results were:
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
The Proxy Playbook: The Complete Guide to Proxy Servers: How to Source, Test, and Scale Residential,... | $29.95 | Buy on Amazon |
| 2 |
|
How to Host your own Web Server | $15.60 | Buy on Amazon |
| Author’s classification | Sources |
|---|---|
| OK | 56 |
| Blocked | 22 |
| Parse failures | 4 |
| Reachable, but nothing parsed | 1 |
| Errors | 2 |
Although 56 sources were classified OK, only 45 returned at least one item. An OK result therefore should not be read as a guarantee that a source produced useful products in that run. The classifications describe the author’s collection process, not a controlled comparison of every retailer or fetching method. Read Anderson’s report on DEV Community.
Why Shopify’s storefront data mattered
The run collected 4,112 discounted products before later filtering. Twenty-one Shopify stores supplied 3,173 of them—more than three quarters of the collected set, according to the author. In the stores tested, the public storefront endpoint /products.json returned product and variant information that included prices, compare-at prices, and per-size availability. That made it easier for this agent to identify discounted products and check variants.
This is an observation about the tested storefronts, not a promise that every Shopify shop exposes the same endpoint or fields in the same usable form. A storefront endpoint also should not be confused with Shopify’s merchant-facing Admin API.
What happened after product collection
The 4,112 collected products were filtered for menswear, clothing or footwear, stock in the author’s size, and a genuine reduction. Anderson reports 941 products confirmed in their size and 452 whose size availability could not be checked. The largest stated removal group was 1,411 products that were the wrong size. These counts reflect the author’s size, filters, and chosen sources; another shopper or source list could produce different results.
A listed price is not proof of in-size stock
Price data and size-level inventory answer different questions. A product can have a sale price and variants without the available inventory for a specific size being readable at the time of collection. Anderson found eight sources that returned products and prices but no readable per-size stock: Nike, JD Sports, Selfridges, Footasylum, Puma, Converse, Clarks, and an unnamed flash-sale site.
For Puma, the report says the size grid and inventory arrived through a later API call. That example shows why a first product response may not contain every detail needed to answer “Can I buy this jacket in my size?” The agent marked items whose size could not be verified as size_unknown and excluded them from alerts rather than treating uncertainty as availability.
Structured data beyond Shopify
The report describes two other examples of data that helped the agent:
- A large retailer’s sale page used a public, search-only Algolia key that the retailer’s own front end used. Anderson reports that the query returned 400 server-filtered items in this run.
- Foot Locker product-page JSON included per-size availability, according to the report.
These are examples from this particular source list and run, not instructions to assume other retailers expose the same data or that an endpoint will remain available.
Why a failed fetch does not automatically mean an anti-bot block
A plain HTTP request that fails, returns an empty result, or cannot retrieve a policy file does not by itself establish why a site failed. Anderson gives examples of several distinct causes:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Rank #2
- JavaScript-dependent pages: a product grid may render in a browser but not appear in the initial plain HTTP response.
- Incorrect route: a bad category path can return 404 even when the site itself is accessible.
- Policy retrieval failure: in five cases, a plain request to
robots.txtreturned 403. The author says browser-style fetching showed*allowed, but later tests still produced no useful menswear results for other reasons. - Access challenge or block: the report describes a retailer that worked for about 30 page loads before Akamai blocked access, as well as sites using DataDome and reCAPTCHA.
Keep these outcomes separate. A 403 from a policy-file request is not, on its own, a confirmed disallow rule; an empty page is not automatically a bot challenge; and a 404 can indicate a wrong path. The report’s conclusion captures the distinction: “Failing to read a policy is not the same as the policy saying no.”
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How the report handled access limits
Anderson says the project identified its client honestly, kept per-site request volumes small, cached results, and backed off for 72 hours after a block. It did not use proxies, CAPTCHA solving, or rotation. Challenge, error, and empty-render responses were treated as blocked rather than worked around. In the author’s words: “No proxies, no captcha solving, no rotating anything.”
This restraint matters to the interpretation of the results: a source marked blocked was not necessarily retried through evasion methods to force a successful response. The report does not establish that its access pattern or classifications will match another agent’s conditions.
Do Shopify Admin API deprecation notices apply to products.json?
No—not automatically. Shopify’s cited documentation concerns the authenticated merchant Admin REST Product resource, a different API context from the public storefront endpoint described in Anderson’s report. Shopify labels the REST Admin API legacy, says product listing, creation, updating, and deletion were deprecated as of REST API 2024-04, and states that new public apps must use the GraphQL Admin API exclusively starting April 1, 2025. Those notices describe Admin API access; they do not prove the behavior or availability of a storefront’s /products.json endpoint. Shopify Admin REST Product resource documentation.
How to interpret the numbers
The results are useful as a case study in what this agent could collect and verify, not as a universal success rate for fashion scraping. The 85 sources were chosen by the author, the measurements came from one run, and the product counts reflect that run’s filters and one shopper’s size. The report also says it was drafted with AI assistance from the author’s notes, logs, and repositories, then reviewed and edited. No independent replication or third-party audit of the measurements is established by the cited evidence.
The most useful distinction for shoppers and builders is between finding a discounted product and verifying that a specific size can actually be bought. Structured data can make that second check possible at some stores, but where size inventory is absent or loaded later, the accurate status is unknown—not available.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
