Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesConvert a cURL command to Python with the Requests library by mapping each cURL option to a named argument: params for query strings, json or data for bodies, headers for headers, auth for credentials, cookies for cookies, files for uploads, and timeout for network limits. Then check the status code, parse the response deliberately, and handle failures explicitly.
This guide builds that translation method from simple GET requests through authentication, sessions, uploads, retries, streaming, and the cases where curl_cffi is a better fit.
Install Requests and make a first call
The Requests documentation currently lists version 2.34.2 and states support for Python 3.10 and newer; verify those version-sensitive details in the official documentation before pinning a deployment.
python -m pip install requestsA safe first request keeps credentials out of source code and sets a finite timeout:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
#1 Best Overall
import os
import requests
url = "https://api.example.com/v1/items"
headers = {"Accept": "application/json"}
response = requests.get(url, headers=headers, timeout=(5, 30))
print(response.status_code)
print(response.headers.get("content-type"))
response.raise_for_status()
if "application/json" in response.headers.get("content-type", "").lower():
print(response.json())
else:
print(response.text[:500])Use environment variables or a secret manager for tokens. Never commit real API keys or print an Authorization header in logs.
Translate a cURL command argument by argument
Consider this representative command:
curl -G "https://api.example.com/search"
-H "Accept: application/json"
-H "Authorization: Bearer $TOKEN"
--data-urlencode "q=python requests"
--data-urlencode "page=2"
--max-time 30The equivalent Requests call is:
import os
import requests
response = requests.get(
"https://api.example.com/search",
params={"q": "python requests", "page": 2},
headers={
"Accept": "application/json",
"Authorization": f"Bearer {os.environ['API_TOKEN']}",
},
timeout=30,
)
response.raise_for_status()
print(response.json())| cURL | Requests | Purpose |
|---|---|---|
-G, --data-urlencode |
params={...} |
URL query parameters; Requests encodes values. |
-H |
headers={...} |
HTTP request headers. |
-d, --data |
data=... |
Form-encoded, raw, or otherwise explicitly prepared body. |
--json |
json={...} |
JSON serialization and the JSON content type. |
-u user:pass |
auth=(user, pass) |
Basic authentication credentials. |
-F field=@file |
files={...} |
Multipart file upload. |
-b |
cookies={...} or a Session |
Send cookies. |
-c cookiejar |
Session cookie persistence | Retain cookies between calls. |
--max-time |
timeout=... |
Connection and read time limits. |
The server contract still decides whether it expects JSON, form data, a particular authentication scheme, redirects, or a specific status code. A mechanical translation cannot replace that contract.
Build requests with the right argument
Query parameters with params
r = requests.get(
"https://api.example.com/items",
params={"tag": ["python", "http"], "limit": 25, "archived": False},
timeout=20,
)
print(r.url) # inspect the encoded URLPassing a dictionary or list of tuples lets Requests perform URL encoding and avoids hand-built query strings.
JSON, form, and raw bodies
# JSON body and Content-Type: application/json
r = requests.post(
"https://api.example.com/items",
json={"name": "demo", "enabled": True},
timeout=20,
)
# Form-encoded body
r = requests.post(
"https://api.example.com/login",
data={"username": "alice", "password": os.environ["PASSWORD"]},
timeout=20,
)
# Raw bytes or text
r = requests.put(
"https://api.example.com/blob",
data=b"binary payload",
headers={"Content-Type": "application/octet-stream"},
timeout=20,
)Do not use data=json.dumps(payload) unless you specifically need manual serialization; json= is less error-prone and sets the expected content type.
Recommended Free Tools
Headers, cookies, files, and authentication
# Headers and explicit cookies
r = requests.get(
"https://api.example.com/profile",
headers={"X-Request-ID": "job-123"},
cookies={"session": os.environ["SESSION_COOKIE"]},
timeout=20,
)
# Basic auth
r = requests.get(
"https://api.example.com/private",
auth=(os.environ["USER"], os.environ["PASSWORD"]),
timeout=20,
)
# Multipart upload
with open("report.pdf", "rb") as f:
r = requests.post(
"https://api.example.com/upload",
files={"document": ("report.pdf", f, "application/pdf")},
data={"description": "Monthly report"},
timeout=(5, 120),
)For bearer tokens, set an Authorization header. Requests also documents Basic and Digest authentication, .netrc, OAuth, and OAuth 2/OpenID Connect integrations; token acquisition, refresh, scopes, and storage remain specific to the identity provider. See the authentication documentation.
Handle responses and HTTP errors deliberately
Useful response attributes are status_code, case-insensitive headers, decoded text, raw content, and json(). Calling raise_for_status() raises HTTPError for unsuccessful HTTP status codes, as described in the quickstart.
import requests
try:
response = requests.get("https://api.example.com/items/42", timeout=(3, 15))
response.raise_for_status()
except requests.exceptions.Timeout:
print("The server took too long to connect or respond")
except requests.exceptions.HTTPError as exc:
print("HTTP failure:", exc, response.text[:300])
except requests.exceptions.RequestException as exc:
print("Transport failure:", exc)
else:
content_type = response.headers.get("content-type", "").lower()
if "application/json" in content_type:
payload = response.json()
else:
payload = response.text
print(payload)A 404 or 422 is an application result, not a network outage. Inspect the response body for the API’s error details and decide whether to correct input, refresh credentials, or stop.
Timeouts, retries, and safe reliability patterns
Requests has no universal implicit deadline you should rely on. A scalar timeout=30 applies one limit; timeout=(5, 30) separates connection establishment from waiting for response bytes. The API reference documents both forms at Requests API.
- Connect timeout: time to establish a connection, including proxy or DNS-related connection work.
- Read timeout: maximum wait for bytes after a connection exists; it is not a total download duration.
- Streaming: use
stream=Trueand iterate overiter_content()for large responses, while retaining a read timeout.
with requests.get(
"https://api.example.com/export",
stream=True,
timeout=(5, 60),
) as r:
r.raise_for_status()
with open("export.bin", "wb") as out:
for chunk in r.iter_content(chunk_size=1024 * 1024):
if chunk:
out.write(chunk)Retry only operations that are safe to repeat, such as idempotent GET requests or POST requests that use an idempotency key. A timeout does not prove the server did not process a write. Use exponential backoff, honor server retry guidance such as Retry-After, and cap attempts. For complex retry policies, configure an HTTPAdapter with urllib3’s retry support rather than blindly looping.
Use a Session for repeated calls
A requests.Session persists cookies and reuses pooled connections, reducing setup overhead for login flows and API clients. The advanced usage guide recommends sessions for this pattern.
import requests
with requests.Session() as session:
session.headers.update({"Accept": "application/json"})
session.auth = ("alice", os.environ["PASSWORD"])
login = session.post(
"https://api.example.com/login",
json={"device": "cli"},
timeout=(5, 20),
)
login.raise_for_status()
data = session.get(
"https://api.example.com/me",
timeout=(5, 20),
)
data.raise_for_status()
print(data.json())Close sessions explicitly or use a context manager. Configure a private CA bundle with the verify parameter or environment settings when required; disabling TLS verification should not be a routine workaround.
When curl_cffi is the better choice
Requests is the default for ordinary API clients and has a familiar, portable interface. curl_cffi offers a Requests-like API backed by curl, with curl-oriented options, sessions, a CLI, and an impersonate parameter documented in its API reference. Its documentation also shows uv run curl-cffi and python -m curl_cffi CLI usage (see the PDF documentation).
| Decision point | Requests | curl_cffi |
|---|---|---|
| Migration effort | Baseline Python API; broad ecosystem. | Similar call surface, with curl-specific controls. |
| Sessions and cookies | Persistent cookies and connection pooling. | Sessions are supported; maintainers advise using one whenever possible. |
| Authentication and proxies | Common auth handlers and proxy configuration. | Requests-like controls plus curl options; verify project-specific behavior. |
| Browser/TLS compatibility | Standard Python TLS and HTTP behavior. | Use when a documented curl impersonation or HTTP/TLS behavior is required. |
| Deployment | Small, conventional dependency for most services. | Additional native/runtime considerations may affect your platform policy. |
Impersonation is a compatibility feature, not permission to bypass a site’s terms, authentication, bot controls, or access restrictions.
Common conversion failures and fixes
“cURL works but Python returns 401”
Compare the complete outgoing request: scheme, host, path, query encoding, capitalization of headers, token value, and cookies. Ensure the token is loaded into the process environment and that you did not accidentally include shell quotes in its value.
“The server says invalid JSON”
Use json=payload, not a Python dictionary in data=. If the endpoint expects form data, use data= and its documented content type instead.
“It hangs forever”
Add a scalar or connect/read timeout. A read timeout is measured between received bytes, so a long streaming response may require a larger read limit and application-level overall deadline.
“SSL: CERTIFICATE_VERIFY_FAILED”
Install the correct trust chain or point Requests at the organization CA bundle. Do not solve production problems with verify=False; that removes certificate verification.
“JSON decoding failed”
Check the status code and Content-Type before calling json(). Error pages, empty 204 responses, and proxy-generated HTML are not JSON merely because the URL is an API URL.
“Uploads are empty or the file is locked”
Open the file in binary mode, keep it open for the request, and let Requests construct multipart boundaries through files=.
Or skip the browser setup
If your Python workflow ultimately needs website screenshots rather than API JSON, ScreenshotNeo provides a single HTTP call. It accepts consent banners before capture and removes more than 60 known consent platforms, newsletter popups, and chat widgets; each cleanup step can be disabled. Bot checks, CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and response headers identify the page verdict and billing status.
Using the same Requests concepts:
import requests
r = requests.get(
"https://api.screenshotneo.com/v1/shot",
params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)See the ScreenshotNeo API documentation for PNG, JPEG, WebP, PDF, device, selector, wait, blocking, caching, signed-link, webhook, and bulk-capture options. Its MCP server lets Claude, Cursor, and other MCP clients call take_screenshot, get_page_info, and capture_pdf. The Free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Create a free ScreenshotNeo account.
Practical checklist
- Move cURL query fields to
params, not string concatenation. - Choose
json,data, orfilesaccording to the server contract. - Keep credentials in environment variables or a secret manager.
- Set connect and read timeouts explicitly.
- Call
raise_for_status()where non-2xx responses are failures. - Inspect content type before parsing JSON.
- Use a Session for repeated calls, shared cookies, and pooled connections.
- Retry only when repeating the operation is safe.
- Use curl_cffi only when its curl or impersonation capabilities are an actual requirement.
Frequently Asked Questions
Does Requests execute a cURL command directly?
No. You translate the command’s options into Requests arguments; this makes the request programmable and testable in Python.
Should I use a Session for one request?
Usually not necessary. A Session pays off when calls share cookies, headers, authentication, or a connection pool.
Is a timeout a total request deadline?
No. The read timeout limits waiting between bytes. Add an application-level deadline if you need a hard wall-clock limit.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchCan curl_cffi impersonation guarantee access to a protected site?
No. It changes client compatibility characteristics and does not grant authorization or override a site’s rules.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
