To fetch data from an HTTP API in Python, send a request to the endpoint, encode any query parameters, set a timeout, check the HTTP status, and then parse the response if it contains JSON. For a dependency-free example, use Python’s standard-library urllib; for a more concise client interface, install and use Requests.
Make a GET request with Python’s standard library
This example uses urllib.request to open a URL, urllib.parse.urlencode to encode query parameters, and json to parse a JSON response. Replace the example endpoint and parameters with those documented by your API.
import json
from urllib.error import HTTPError, URLError
from urllib.parse import urlencode
from urllib.request import urlopen
endpoint = "https://api.example.com/items"
params = {"q": "blue sky", "limit": 10}
url = f"{endpoint}?{urlencode(params)}"
try:
with urlopen(url, timeout=10) as response:
status = response.status
body = response.read()
charset = response.headers.get_content_charset() or "utf-8"
if not 200 <= status < 300:
raise RuntimeError(f"Unexpected HTTP status: {status}")
data = json.loads(body.decode(charset))
print(data)
except HTTPError as exc:
print(f"The server returned HTTP {exc.code}: {exc.reason}")
except URLError as exc:
print(f"Could not reach the server: {exc.reason}")
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
print(f"The response could not be decoded as JSON: {exc}")urlopen returns a response-like object whose body is bytes. The context manager closes it when the block ends. The example reads the response charset from its headers, falling back to UTF-8 if none is supplied, then decodes those bytes before calling json.loads. Python documents urllib.request as its URL-opening interface and provides urlencode for query-string encoding (urllib.request reference; urllib package overview).
Why encode query parameters?
Do not concatenate raw user input into a URL. Values such as blue sky contain characters that need encoding in a query string. Passing a mapping to urlencode handles that encoding for this common case. The API’s documentation determines the endpoint, parameter names, and whether the request should use GET or another method. Python’s urllib HOWTO covers constructing requests and encoding form data.
#1 Best Overall
Use Requests for a higher-level interface
Requests is a separate package, not part of Python’s standard library. Install it in your project environment with python -m pip install requests, then pass query parameters as a mapping with params=.
import requests
url = "https://api.example.com/items"
params = {"q": "blue sky", "limit": 10}
try:
response = requests.get(url, params=params, timeout=10)
response.raise_for_status()
data = response.json()
print(data)
except requests.exceptions.Timeout:
print("The request timed out")
except requests.exceptions.HTTPError as exc:
print(f"The server returned an unsuccessful HTTP status: {exc}")
except requests.exceptions.RequestException as exc:
print(f"The request failed: {exc}")
except requests.exceptions.JSONDecodeError as exc:
print(f"The response was not valid JSON: {exc}")Requests also offers json= when sending a JSON request body, and Response.json() for decoding a JSON response. The example uses raise_for_status() before parsing so an HTTP error is handled as an HTTP error rather than mistaken for successful data. Requests 2.34.2 documents its query parameter, JSON, status, and timeout behavior in its Quickstart.
Keep HTTP errors separate from JSON errors
An HTTP response and its body are separate things. A server can return an error status with a body that happens to contain valid JSON; valid JSON does not mean the request succeeded. Requests cautions: “The success of the call to r.json() does not indicate the success of the response.” Check the status with raise_for_status() or an expected status code before treating decoded content as success.
The reverse distinction matters too: a request can receive a successful HTTP status but still fail JSON decoding if the body is empty, malformed, or not JSON. Parse JSON only when the endpoint’s documentation says the response is JSON, and handle decoding errors separately from network and status failures.
Free tools Windows power users keep installed
One-click scans. No signup required.
Set a timeout and choose the client
A timeout prevents your program from waiting indefinitely on a network operation. Both examples set one explicitly: urlopen(..., timeout=10) for urllib and requests.get(..., timeout=10) for Requests. Requests notes that its timeout is not a wall-clock limit for downloading the entire response; it governs periods during which no data is received. Its guidance is: “Nearly all production code should use this parameter in nearly all requests.”
| Consideration | urllib |
Requests |
|---|---|---|
| Dependency | Included in Python’s standard library. | Separate package that must be installed. |
| Query parameters | Build the query string with urllib.parse.urlencode. |
Pass a mapping with params=. |
| JSON response | Read bytes, decode to text, then use json.loads. |
Use Response.json(). |
| Status handling | Handle HTTP responses through HTTPError and check expected status codes. |
Call raise_for_status() or check the status code. |
| Timeout | Pass timeout= to urlopen. |
Pass timeout= to the request; it is not a total-download deadline. |
Choose urllib when you want a basic request without an added dependency. Choose Requests when its concise parameter handling, response helpers, and higher-level interface suit your project; Python’s current urllib documentation specifically recommends Requests for a higher-level HTTP client interface.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
