How to Fix ChunkedEncodingError in Python requests
Find where the body gets cut, then retry or resume the download with code for Python, Node.js and cURL.
Updated
TL;DR
ChunkedEncodingError means Python requests got the response headers, but the connection closed before the body was complete, so the read broke partway. The fastest fix is to catch the error and send the GET again, resuming large files with a Range header instead of starting over.
What a broken body looks like in each client
Byte counts show where the body stopped, so yours will differ.
| Where | What you see |
|---|---|
| Python requests with urllib3 2.x, cut inside a chunk | requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(500 bytes read, 524 more expected)', IncompleteRead(500 bytes read, 524 more expected)) |
| Python requests with urllib3 2.x, cut between chunks | requests.exceptions.ChunkedEncodingError: Response ended prematurely |
| Python requests with urllib3 1.26, cut between chunks | requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read)) |
| pip 24.0 and 26.1, package download cut (after ERROR: Exception:) | pip._vendor.urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(500 bytes read, 524 more expected)', IncompleteRead(500 bytes read, 524 more expected)) |
| httpx 0.27 | httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) |
| Node.js 20 and 24 fetch(), reading the body | TypeError: terminated ... [cause]: SocketError: other side closed |
| curl | curl: (18) transfer closed with outstanding read data remaining |
Why this happens
The connection closed before the response body reached its end.
A chunked body ends with a zero-size chunk, and a Content-Length body ends at its stated size. When the server, a proxy or the network closes the connection sooner, urllib3 flags the read as broken.
Diagnose your ChunkedEncodingError first
Each check narrows down where the body gets cut: at the site, on the route or in your own loop.
Download the same URL in a browser on this computer; if it also stops partway, the site or your connection ends it, not your script.
Run curl --http1.1 -sS -o /dev/null with the same proxy and URL; curl: (18) means the body is cut before it ever reaches Python.
Repeat the download once through the proxy and once without it; if only the proxied run breaks, the drop happens on the proxy route.
Compare byte counts in the error across runs; an identical count each time means the server stops at one point, so retries will not help.
Log the seconds between chunks in your iter_content loop; if errors follow long gaps, the server stopped waiting for a slow reader.
Solutions ranked by effectiveness
Start with a plain retry; the other two cover large files and loops that read slowly.
- Most common fix
Catch the error and request again
Applies when cuts come and go. Catch ChunkedEncodingError, wait a little longer each time and repeat the GET on a new connection; curl repeats its error 18 only with --retry-all-errors.
# pip install requests import time import requests proxy = "http://USERNAME:PASSWORD@proxy.geonode.io:9000" for attempt in range(1, 5): try: r = requests.get("https://example.com/data.json", proxies={"http": proxy, "https": proxy}, timeout=60) print(r.status_code, len(r.content), "bytes") break except requests.exceptions.ChunkedEncodingError: if attempt == 4: raise time.sleep(2 ** attempt) - Check next
Resume large files from the last byte
Applies to big downloads that break after many megabytes. When the response carries Accept-Ranges: bytes, ask only for the missing part instead of fetching the whole file again.
After a cut, send the GET again with Range: bytes=<bytes already saved>-.
Append the new body to your file only if the answer is 206 Partial Content.
If the server answers 200, it ignored Range; rewrite the file from the start.
With curl, use -C -; before curl 8.17.0, --retry restarts it, so rerun instead.
- For developers
Keep the read loop moving
Applies when cuts follow slow work between chunks. A server stops sending to a client that takes nothing for too long; nginx's send_timeout, for example, defaults to 60 seconds.
Write each chunk to disk inside the loop and process the file afterwards.
Move parsing, database writes and API calls out of the iter_content loop.
When work must happen live, put chunks on a queue that another thread drains.
Stop the ChunkedEncodingError from coming back
Habits that catch short downloads early and keep a cut file from passing as finished.
Save to a .part file and rename it only after the last chunk arrives.
Check the file's size or hash against the value the publisher lists before you use it.
Pass a read timeout such as timeout=(10, 60), so a stalled body fails instead of hanging.
Log the inner exception's class and byte counts per URL, so hosts that cut often stand out.
Related errors
Learn more
FAQ
One exit IP per download
Sticky ports keep one IP for a lifetime you set, 3 minutes to 24 hours.
