Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Short answer: Cloudscraper gives Python a Requests-style session for sites that present certain Cloudflare challenges, but it is not a universal bypass. Create a scraper session, send an authorized request, inspect the response, and stop when the site continues to require a challenge. Cloudflare uses several different challenge mechanisms, and a successful response on one site does not establish compatibility with another.

What Cloudscraper is—and what it is not

Cloudscraper is a third-party Python package whose documented workflow resembles the requests library. You create a scraper session with cloudscraper.create_scraper(), then call methods such as get() or post(). The session manages the request flow described by the project documentation.

That familiar interface only describes how you send requests. It does not promise that a particular Cloudflare-protected site will grant access, nor does it make an automated client equivalent to a normal browser. Use the examples only with a site you own, a system for which you have written permission, or data the operator explicitly permits you to retrieve.

When this approach is appropriate

  • You are testing a site you administer and need to reproduce a normal HTTP request in a controlled environment.
  • The owner has authorized automated retrieval and has told you which paths, rates and data are allowed.
  • You are experimenting against a local or staging copy rather than trying to defeat a production access control.

When to choose another route

If the site offers an API, data export or documented feed, use that first. Those interfaces are more stable, easier to monitor and clearer about permitted volume. A persistent challenge is an access-control signal, not an invitation to keep changing clients until one slips through.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
#1 Best Overall
Sale
TP-Link ER605, Wired Gigabit VPN Router
  • 【Five Gigabit Ports】1 Gigabit WAN Port plus 2 Gigabit WAN/LAN Ports plus 2 Gigabit LAN Port. Up to 3 WAN ports optimize bandwidth usage through one device.
  • 【One USB WAN Port】Mobile broadband via 4G/3G modem is supported for WAN backup by connecting to the USB port. For complete list of compatible 4G/3G modems, please visit TP-Link website.
  • 【Abundant Security Features】Advanced firewall policies, DoS defense, IP/MAC/URL filtering, speed test and more security functions protect your network and data.
  • 【Highly Secure VPN】Supports up to 20× LAN-to-LAN IPsec, 16× OpenVPN, 16× L2TP, and 16× PPTP VPN connections.
  • Security - SPI Firewall, VPN Pass through, FTP/H.323/PPTP/SIP/IPsec ALG, DoS Defence, Ping of Death and Local Management. Standards and Protocols IEEE 802.3, 802.3u, 802.3ab, IEEE 802.3x, IEEE 802.1q

Install Cloudscraper and make the minimal request

Install the package in the virtual environment used by your project:

python -m pip install cloudscraper

The documented pattern is:

import cloudscraper

scraper = cloudscraper.create_scraper()
response = scraper.get("https://example.com")

print(response.status_code)
print(response.text[:500])

Replace https://example.com with a domain you control or are authorized to access. The package documentation describes create_scraper() as returning a session-like object and shows Requests-style calls such as get() and post(). This snippet is a usage illustration; it is not a guarantee that the target will return content or that a challenge will be solved.

A safer diagnostic version

Before parsing HTML, record the status, content type and a short prefix. A challenge page can be valid HTML while still denying the requested resource.

import cloudscraper

url = "https://example.com/authorized-test"
scraper = cloudscraper.create_scraper()
response = scraper.get(url, timeout=30)

print("status:", response.status_code)
print("content-type:", response.headers.get("content-type"))
print("final URL:", response.url)
print(response.text[:500])

Use a timeout so a failed origin does not leave a worker waiting indefinitely. Keep request rates within the limits agreed with the site owner.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Understand the Cloudflare challenge you are seeing

Cloudflare documentation states: “Challenges can be issued in three primary ways depending on which Cloudflare products or features are in use.” In practice, the product configuration determines what your client receives.

Rank #2
GL.iNet GL-SFT1200 Opal Travel Router, AC1200 Dual-Band Wi-Fi
  • 【AC1200 Dual-band Wireless Router】Simultaneous dual-band with wireless speed up to 300 Mbps (2.4GHz) + 867 Mbps (5GHz). 2.4GHz band can handles some simple tasks like emails or web browsing while bandwidth intensive tasks such as gaming or 4K video streaming can be handled by the 5GHz band.*Speed tests are conducted on a local network. Real-world speeds may differ depending on your network configuration.*
  • 【Easy Setup】Please refer to the User Manual and the Unboxing & Setup video guide on Amazon for detailed setup instructions and methods for connecting to the Internet.
  • 【Pocket-friendly】Lightweight design(145g) which designed for your next trip or adventure. Alongside its portable, compact design makes it easy to take with you on the go.
  • 【Full Gigabit Ports】Gigabit Wireless Internet Router with 2 Gigabit LAN ports and 1 Gigabit WAN ports, ideal for lots of internet plan and allow you to connect your wired devices directly.
  • 【Keep your Internet Safe】IPv6 supported. OpenVPN & WireGuard pre-installed, compatible with 30+ VPN service providers. Cloudflare encryption supported to protect the privacy.

Interstitial challenge pages

WAF rules and Bot Fight modes can issue an interstitial challenge page. Your first response may therefore be a challenge document rather than the application page. A Requests-style session can follow the documented flow for mechanisms the project supports, but the response still depends on the zone’s rules and the signals Cloudflare evaluates.

JavaScript Detections

Cloudflare Bot Management can use JavaScript Detections. This is different from assuming that every challenge is the same browser-style interstitial. A client that handles one mechanism may not satisfy another configuration.

Turnstile widgets

Turnstile is an embedded widget. It is an explicit verification component in the page, not merely a redirect that a generic HTTP session can treat as a normal response. If the site requires a human interaction or a site-specific token, follow the operator’s supported integration rather than attempting to automate around it.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

DDoS protection and Under Attack Mode

HTTP DDoS protection can issue any challenge. Under Attack Mode uses Managed Challenge. The visible page and the required signals can therefore change when the administrator changes protection settings.

Build a complete, authorized retrieval script

For a permitted job, separate transport, validation and parsing. This makes it obvious when you received a challenge rather than the data you expected.

Rank #3
Sale
ASUS RT-AX1800S Dual Band WiFi 6 Extendable Router, Subscription-Free Network Security, Parental Control, Built-in VPN, AiMesh Compatible, Gaming & Streaming, Smart Home
  • New-Gen WiFi Standard – WiFi 6(802.11ax) standard supporting MU-MIMO and OFDMA technology for better efficiency and throughput.Antenna : External antenna x 4. Processor : Dual-core (4 VPE). Power Supply : AC Input : 110V~240V(50~60Hz), DC Output : 12 V with max. 1.5A current.
  • Ultra-fast WiFi Speed – RT-AX1800S supports 1024-QAM for dramatically faster wireless connections
  • Increase Capacity and Efficiency – Supporting not only MU-MIMO but also OFDMA technique to efficiently allocate channels, communicate with multiple devices simultaneously
  • 5 Gigabit ports – One Gigabit WAN port and four Gigabit LAN ports, 10X faster than 100–Base T Ethernet.
  • Commercial-grade Security Anywhere – Protect your home network with AiProtection Classic, powered by Trend Micro. And when away from home, ASUS Instant Guard gives you a one-click secure VPN.
import sys
import cloudscraper

URL = "https://example.com/authorized-data"


def fetch(url: str) -> str:
    scraper = cloudscraper.create_scraper()
    response = scraper.get(url, timeout=30)
    response.raise_for_status()

    content_type = response.headers.get("content-type", "")
    if "text/html" not in content_type.lower():
        raise RuntimeError(f"Unexpected content type: {content_type}")

    body = response.text
    markers = ("challenge", "turnstile", "cf-chl", "attention required")
    if any(marker in body.lower() for marker in markers):
        raise RuntimeError("The response appears to be a Cloudflare challenge, not the requested page")
    return body


if __name__ == "__main__":
    try:
        html = fetch(URL)
    except Exception as exc:
        print(f"Request stopped: {exc}", file=sys.stderr)
        raise SystemExit(1)
    print(html[:500])

The marker check is only a diagnostic safeguard. It is not a Cloudflare verifier and should not be treated as proof that a page is safe to parse. Your application should use selectors and schema checks appropriate to the site you are authorized to access.

Configuration options documented by the project

The Cloudscraper project documents options for selecting an interpreter, adding a delay, enabling debug output and configuring CAPTCHA-solver integrations, along with other session settings. These are project-level controls, not guaranteed fixes for a Cloudflare decision.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Interpreter and delay settings

If your authorized test requires a particular interpreter or delay, configure it exactly as documented by the installed version and record the setting in your test logs. A delay can reduce request burstiness, but it cannot make an unauthorized workload acceptable or ensure that a challenge will pass.

Debug output

Enable the project’s documented debug mode while diagnosing a staging system. Capture logs without publishing cookies, authorization headers or page contents. Disable verbose output in production unless you have a retention and redaction policy.

CAPTCHA and interactive verification

A CAPTCHA-solver option does not grant permission to solve a site’s challenge. Do not use it to defeat an operator’s controls. If an authorized integration genuinely needs a human verification step, ask the site owner for the supported API or test configuration instead.

Rank #4
Sale
GL.iNet GL-BE3600 Slate 7 Wi-Fi 7 Travel Router Touchscreen 2.5G
  • 【DUAL BAND WIFI 7 TRAVEL ROUTER】Products with US, UK, EU, AU Plug; Dual band network with wireless speed 688Mbps (2.4G)+2882Mbps (5G); Dual 2.5G Ethernet Ports (1x WAN and 1x LAN Port); USB 3.0 port.
  • 【NETWORK CONTROL WITH TOUCHSCREEN SIMPLICITY】Slate 7’s touchscreen interface lets you scan QR codes for quick Wi-Fi, monitor speed in real time, toggle VPN on/off, and switch providers directly on the display. Color-coded indicators provide instant network status updates for Ethernet, Tethering, Repeater, and Cellular modes, offering a seamless, user-friendly experience.
  • 【OpenWrt 23.05 FIRMWARE】The Slate 7 (GL-BE3600) is a high-performance Wi-Fi 7 travel router, built with OpenWrt 23.05 (Kernel 5.4.213) for maximum customization and advanced networking capabilities. With 512MB storage, total customization with open-source freedom and flexible installation of OpenWrt plugins.
  • 【VPN CLIENT & SERVER】OpenVPN and WireGuard are pre-installed, compatible with 30+ VPN service providers (active subscription required). Simply log in to your existing VPN account with our portable wifi device, and Slate 7 automatically encrypts all network traffic within the connected network. Max. VPN speed of 100 Mbps (OpenVPN); 540 Mbps (WireGuard). *Speed tests are conducted on a local network. Real-world speeds may differ depending on your network configuration.*
  • 【PERFECT PORTABLE WIFI ROUTER FOR TRAVEL】The Slate 7 is an ideal portable internet device perfect for international travel. With its mini size and travel-friendly features, the pocket Wi-Fi router is the perfect companion for travelers in need of a secure internet connectivity on the go in which includes hotels or cruise ships.

Sessions, IP addresses and challenge loops

Keep one logical session together when the site owner has approved the workflow. Cloudflare explicitly warns that a Managed Challenge solve request coming from an IP different from the IP that made the original challenge request is invalid and may produce a challenge loop. Do not assume that moving a token between machines, changing proxies mid-session or replaying it later will work.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Use a stable, approved network path for the duration of a request flow.
  • Do not share cookies or challenge tokens between unrelated workers.
  • Record timestamps and response headers so the site administrator can correlate failures.
  • Stop rather than repeatedly retrying when the response remains a challenge.

Troubleshooting: symptoms, causes and responsible fixes

Symptom Likely explanation Recommended action
HTTP 403 or a challenge HTML page The zone’s rules require a challenge your request did not satisfy. Stop retries; check the site’s API, export or authorization instructions. If you administer the zone, inspect the matching Cloudflare rule.
Repeated challenge loop The session, IP or required browser signals changed; Cloudflare documents IP changes as invalid for a Managed Challenge solve. Keep the approved session and network stable, or ask the owner for an integration path. Do not circulate challenge tokens.
HTTP 429 The origin or protection layer is rate-limiting requests. Honor the published limit, reduce volume and use backoff agreed with the operator. An API may provide a better quota model.
Timeout or empty response The origin, network or protection layer did not complete the request. Use a finite timeout, log the failure, retry only under an approved policy and verify the service status with its owner.
HTML parses but expected fields are absent You received a challenge, consent page, login page or redesigned document. Validate content type and required selectors before storing data; send a sample to the site administrator.
CAPTCHA or Turnstile appears The site requires an interactive or site-specific verification flow. Use the documented API or obtain explicit owner guidance; do not automate around the control.

Cloudscraper versus an official API

Consideration Cloudscraper session Official API or export
Explicit support Depends on the site’s current Cloudflare and application configuration. Provided and documented by the site owner.
Challenge handling May encounter interstitials, JavaScript Detection, Turnstile or Managed Challenge. Usually defines authentication, quotas and response formats directly.
Maintenance Can require changes when protection rules or page markup change. Still requires version monitoring, but the contract is clearer.
Permitted volume Must be agreed with the operator; a Requests-like client does not change that requirement. Quota and rate limits are normally stated by the provider.
Structured data You may need to parse changing HTML. Structured fields are generally the intended consumption format.

When a challenge blocks an authorized project, the official API or a documented data export is the recommended next step. Cloudflare describes Managed Challenge as a way to limit scraping attacks, and its administrator guidance says API calls that should not be challenged should be excluded from challenge actions.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Performance, reliability and cost considerations

No independent performance, success-rate or timing study establishes that Cloudscraper works across Cloudflare-protected sites. Treat the package README’s broad capability statements as maintainer documentation, not a service-level guarantee.

  • Performance: measure your own authorized workload, including DNS, TLS, challenge and origin time, rather than assuming a fixed latency.
  • Reliability: persist status codes and response classifications, and alert when the page shape changes.
  • Cost: account for your compute, bandwidth, parsing and monitoring costs. If the owner offers an API, compare its quota or subscription price with the operational cost of HTML retrieval.
  • Freshness: cache only when the site’s terms and data semantics permit it; otherwise request the minimum frequency needed.

Or skip the browser setup

If your actual requirement is a clean image or PDF of a page you are authorized to capture—not HTML scraping—ScreenshotNeo provides a single HTTP endpoint. It accepts consent banners before capture and removes more than 60 known consent platforms, newsletter popups and chat widgets; each cleanup step can be disabled. Bot checks, blank pages, timeouts, failed loads and cache hits are not billed, and the response identifies the page verdict and billing status in X-Page-Verdict and X-Billed headers. Its MCP server provides take_screenshot, get_page_info and capture_pdf tools for Claude, Cursor and other MCP clients.

Use the documented request pattern (adapt the target URL only):

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

See the ScreenshotNeo documentation for the full option set, including full-page capture, CSS selectors, device presets, custom JavaScript, waits, blocking rules, PDF settings, caching and signed links. One thousand shots per month are free with no card; paid plans start at $5 for 3,000 shots. Create a free ScreenshotNeo account.

Best Value
Sale
TP-Link Dual-Band AX3000 Wi-Fi 6 Wireless Gigabit Internet Router for Home
  • Next-Gen Gigabit Wi-Fi 6 Speeds: 2402 Mbps on 5 GHz and 574 Mbps on 2.4 GHz bands ensure smoother streaming and faster downloads; support VPN server and VPN client¹
  • A More Responsive Experience: Enjoy smooth gaming, video streaming, and live feeds simultaneously. OFDMA makes your Wi-Fi stronger by allowing multiple clients to share one band at the same time, cutting latency and jitter.²
  • Expanded Wi-Fi Coverage: 4 high-gain external antennas and Beamforming technology combine to extend strong, reliable, Wi-Fi throughout your home.
  • Improved Battery Life: Target Wake Time helps your devices to communicate efficiently while consuming less power.
  • Improved Cooling Design: No heat ups, no throttles. A larger heat sink and redefined case design cools the WiFi 6 system and enables your network to stay at top speeds in more versatile environments.

Further reading

Ryan Mitchell’s Web Scraping with Python, 3rd Edition, published by O’Reilly, covers general mechanics such as making Python requests, handling responses and interacting with sites automatically. It is a general web-scraping book, not a Cloudscraper- or Cloudflare-challenge manual.

Frequently Asked Questions

Does Cloudscraper work with every Cloudflare-protected website?

No. Cloudflare uses multiple challenge products and site-specific rules. The package’s Requests-like interface does not establish compatibility with any particular domain.

What should I do if an authorized request is challenged repeatedly?

Stop automated retries and use the site’s published API, export or owner-approved integration. If you administer the site, inspect the matching Cloudflare rule and ensure legitimate API calls are configured appropriately.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Can I move a Cloudflare challenge token to another server?

Cloudflare documents that a Managed Challenge solve from an IP different from the original challenge request is invalid and may cause a loop. Keep the approved session and network consistent instead.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.