httpx is the HTTP client most new Python projects reach for when they outgrow requests, mainly because it ships one API for sync and async code. This guide shows how to wire an httpx proxy into both Client and AsyncClient, authenticate against a rotating residential endpoint, pin a sticky session, add SOCKS5, tune timeouts and retries, and diagnose the three errors you will actually see. By the end you will have a reusable client that routes through a proxy and can prove its exit IP.
Examples target httpx 0.27 or newer. If you are on an older release, note that the proxies= keyword was renamed to proxy= and mounts= in httpx 0.26; everything below uses the current names.
Two ways to configure an httpx proxy: proxy and mounts
httpx gives you one simple switch and one precise one. proxy= takes a single URL and routes every request through it. mounts= maps URL patterns to transports, and each transport can carry its own proxy, retry count and connection limits.
| Option | Scope | When to use |
|---|
proxy="http://..." | Whole client, all schemes | One proxy for everything, which is the usual case with a rotating gateway |
mounts={"https://": HTTPTransport(proxy=...)} | Per URL pattern | Different proxies per scheme or host, or bypassing the proxy for internal services |
httpx.get(url, proxy=...) | Single top-level call | Quick checks and scripts without a long-lived client |
The simple form looks like this:
import httpx
PROXY = "http://USERNAME:PASSWORD@premium-residential.geonode.com:9000"
with httpx.Client(proxy=PROXY, timeout=20.0) as client:
r = client.get("https://api.ipify.org?format=json")
print(r.status_code, r.json())
Port 9000 is the HTTP(S) rotating gateway, so each new connection gets a different residential IP. Credentials sit in the URL as Basic auth, and httpx turns them into a Proxy-Authorization header without any extra code.
The per-scheme routing gotcha
With mounts you can give http:// and https:// targets separate transports. The trap is the scheme of the proxy URL itself. For an https:// target httpx issues CONNECT to the proxy and then negotiates TLS with the origin inside that tunnel. The hop to the proxy is plain HTTP, so the proxy URL on the https:// mount should normally start with http://, not https://. Writing https:// there tells httpx to wrap the proxy hop itself in TLS, which most gateways do not serve, and you get a handshake failure that looks unrelated to proxies.
import httpx
PROXY = "http://USERNAME:PASSWORD@premium-residential.geonode.com:9000"
mounts = {
"http://": httpx.HTTPTransport(proxy=PROXY),
"https://": httpx.HTTPTransport(proxy=PROXY), # http:// scheme is correct here
# internal services stay direct: mount a transport without a proxy
"all://*.internal.example": httpx.HTTPTransport(),
}
with httpx.Client(mounts=mounts) as client:
print(client.get("https://api.ipify.org?format=json").json())
More specific patterns win over general ones, so the all://*.internal.example mount bypasses the proxy for that domain while everything else is routed.
Authentication in the URL
Geonode credentials are plain username and password, so the userinfo form http://user:pass@host:port is all you need. Two practical notes. First, if the password contains characters such as @, : or /, percent-encode it with urllib.parse.quote before building the URL. Second, keep secrets out of source control by reading them from the environment:
import os
from urllib.parse import quote
import httpx
user = os.environ["GEONODE_USER"]
password = quote(os.environ["GEONODE_PASS"], safe="")
proxy = f"http://{user}:{password}@premium-residential.geonode.com:9000"
client = httpx.Client(proxy=proxy)
Everything that follows adds targeting and session options by extending the username, not the URL, so the same pattern keeps working.
Sync Client versus AsyncClient
The async client takes the same proxy= keyword. The only difference appears when you move to mounts: AsyncClient requires httpx.AsyncHTTPTransport, not httpx.HTTPTransport. Passing the sync transport to an async client raises on the first request.
import asyncio
import httpx
PROXY = "http://USERNAME:PASSWORD@premium-residential.geonode.com:9000"
async def fetch(client: httpx.AsyncClient, url: str) -> str:
r = await client.get(url)
r.raise_for_status()
return r.json()["ip"]
async def main() -> None:
transport = httpx.AsyncHTTPTransport(proxy=PROXY, retries=2)
async with httpx.AsyncClient(
mounts={"all://": transport}, timeout=httpx.Timeout(10.0, connect=5.0)
) as client:
urls = ["https://api.ipify.org?format=json"] * 5
ips = await asyncio.gather(*(fetch(client, u) for u in urls))
print(ips)
asyncio.run(main())
Because port 9000 rotates per connection, the five concurrent requests usually print several different IPs. That is the expected behaviour for a rotating pool; if you need them to agree, use a sticky session (below).
SOCKS5 through httpx[socks]
SOCKS support in an httpx proxy setup is optional. Install it with pip install "httpx[socks]", which adds the socksio package, then swap the scheme. Geonode's SOCKS5 rotating port is 11000:
import httpx
PROXY = "socks5://USERNAME:PASSWORD@premium-residential.geonode.com:11000"
with httpx.Client(proxy=PROXY) as client:
print(client.get("https://api.ipify.org?format=json").json())
Without the extra installed, httpx raises an ImportError that mentions socksio as soon as it sees a socks5:// proxy URL. For ordinary scraping the HTTP ports are simpler to debug because 407 responses are readable.
Environment variables and trust_env
With the default trust_env=True, httpx reads HTTP_PROXY, HTTPS_PROXY, ALL_PROXY and NO_PROXY and applies them automatically. That is convenient on a workstation and a source of confusion in CI, where a corporate proxy in the environment silently rewrites your routing. If you set the proxy in code, consider disabling the environment lookup so behaviour is identical everywhere:
import httpx
client = httpx.Client(
proxy="http://USERNAME:PASSWORD@premium-residential.geonode.com:9000",
trust_env=False,
)
An explicit proxy= already takes precedence over the environment; trust_env=False also stops httpx from reading SSL_CERT_FILE, so set it deliberately.
Timeouts and transport-level retries
A proxy adds a hop, so connect time rises and the default 5-second timeout is sometimes tight for residential exits. httpx.Timeout lets you set a total budget with a separate connect limit, and HTTPTransport(retries=...) retries failed connection attempts before the request is even sent.
import httpx
PROXY = "http://USERNAME:PASSWORD@premium-residential.geonode.com:9000"
transport = httpx.HTTPTransport(proxy=PROXY, retries=2)
timeout = httpx.Timeout(15.0, connect=8.0)
with httpx.Client(mounts={"all://": transport}, timeout=timeout) as client:
r = client.get("https://api.ipify.org?format=json")
print(r.json())
Transport retries cover ConnectError and similar failures while establishing the connection. They do not retry on status codes, and they do not retry after bytes have been sent, so a 502 from the gateway or a target-site 429 still reaches your code. Handle those in a small loop of your own, where you can also decide whether to change the session id.
Sticky sessions with the Geonode username grammar
Rotating on every connection is wrong for multi-step flows such as login, paginate, download. For those your httpx proxy must keep the same exit IP across requests. Geonode encodes this in the username and serves it on the HTTP(S) sticky port 10000:
USER[-type-residential|datacenter|mix][-country-xx][-state-name|-city-name][-asn-N][-session-XXXXXXXX][-lifetime-SECONDS]
The session id is 8 alphanumeric characters that you generate; the lifetime is 180 to 86400 seconds (24 hours). State and city cannot be combined, and asn requires a country.
import secrets
import httpx
def sticky_client(country: str = "us", lifetime: int = 600) -> httpx.Client:
session_id = secrets.token_hex(4) # 8 hex characters
username = f"USERNAME-country-{country}-session-{session_id}-lifetime-{lifetime}"
proxy = f"http://{username}:PASSWORD@premium-residential.geonode.com:10000"
return httpx.Client(proxy=proxy, timeout=httpx.Timeout(15.0, connect=8.0))
with sticky_client() as client:
first = client.get("https://api.ipify.org?format=json").json()["ip"]
second = client.get("https://api.ipify.org?format=json").json()["ip"]
print(first == second, first)
A new session_id means a new IP, so rotating "by hand" is simply creating a fresh client. Keep sticky lifetimes as short as the job needs.
Verify the exit IP
Every example above hits https://api.ipify.org?format=json, which returns {"ip": "..."} for whichever address reached it. Compare that with a direct call (no proxy) once, and you have a two-line smoke test that proves the proxy is in the path. Do this before debugging anything else: many "the httpx proxy is not working" reports are requests that never used the proxy because an environment variable or a mount pattern took over.
Handling 407 ProxyError and ConnectError
Three failures account for nearly all httpx proxy issues. The table maps each to its cause and the fix.
| Symptom | Exception | Usual cause | Fix |
|---|
| Response status 407 | httpx.ProxyError (raised during tunnel setup for https targets) or a plain 407 response for http targets | Wrong credentials, or a username option the gateway rejects | Check the username grammar; percent-encode the password |
| Tunnel refused, 502/503 from proxy | httpx.ProxyError | Gateway could not open the tunnel, or the port is wrong for the protocol | Confirm 9000/10000 for HTTP(S), 11000/12000 for SOCKS5 |
| Cannot reach proxy host | httpx.ConnectError | DNS, firewall, or wrong hostname | Resolve the host manually; raise connect timeout |
| Request hangs then fails | httpx.ConnectTimeout / httpx.ReadTimeout | Slow residential exit or blocked target | Raise timeout, retry with a new session id |
import httpx
PROXY = "http://USERNAME:PASSWORD@premium-residential.geonode.com:9000"
def get_via_proxy(url: str) -> httpx.Response | None:
try:
with httpx.Client(proxy=PROXY, timeout=httpx.Timeout(15.0, connect=8.0)) as client:
r = client.get(url)
if r.status_code == 407:
raise RuntimeError("proxy rejected credentials (407)")
return r
except httpx.ProxyError as e:
print("proxy refused tunnel:", e)
except httpx.ConnectError as e:
print("cannot connect to proxy host:", e)
except httpx.TimeoutException as e:
print("timed out:", type(e).__name__)
return None
resp = get_via_proxy("https://api.ipify.org?format=json")
print(resp.json() if resp else "failed")
All httpx exceptions inherit from httpx.HTTPError, so a single broad except works for logging, but separating ProxyError from ConnectError tells you whether to look at credentials or at the network.
Common mistakes
Using https:// in the proxy URL
As covered above, the proxy URL scheme describes the hop to the proxy. Use http:// for the HTTP(S) ports even when the target is https://. Reserve https:// for gateways that explicitly advertise TLS to the proxy.
Mixing HTTPTransport into an AsyncClient
mounts on AsyncClient must use AsyncHTTPTransport. The sync transport is accepted at construction time and fails at request time with an unhelpful error, so this one is easy to miss in tests that never send a request.
Expecting one IP from the rotating port
Port 9000 rotates per connection. If two requests in a row must share an IP, use port 10000 with a -session- id in the username. Reusing a Client does not pin the IP by itself because connections in the pool can be replaced.
Letting the environment override your code
A stray HTTPS_PROXY in a Docker image or CI runner will route traffic you thought was direct, and NO_PROXY can exclude hosts you expected to be proxied. Set trust_env=False when proxy routing must be deterministic.
Next step
Once the client above returns a residential IP and a sticky session holds across requests, the remaining variable is the pool behind it. Geonode's residential proxies expose the endpoints used in this guide: 2.5M IPs online, 30–50M unique IPs rotating monthly across 195+ countries, with country, state, city and ASN targeting via the username grammar and sticky sessions up to 24 hours. Pricing starts at $0.27/GB at volume, the entry plan is 10 GB for $7.92/month, and a 3-day trial costs $5. If you are building a crawler on top of this, the web scraping use case outlines how to pair proxies with retry logic, and rotating datacenter proxies are the cheaper option for targets that do not block datacenter ranges.
FAQ
How do I set an httpx proxy for all requests?
Pass the proxy URL to the client constructor: httpx.Client(proxy="http://user:pass@host:port"). Every request made through that client, for both http:// and https:// targets, is routed through the proxy. The same keyword works on httpx.AsyncClient and on top-level helpers such as httpx.get(). Credentials in the URL are sent as Proxy-Authorization automatically.
What is the difference between proxy and mounts in httpx?
proxy is a shortcut that routes everything through one proxy. mounts maps URL patterns such as "http://" or "https://" to transports, each of which can carry its own proxy, retries and limits. Use mounts when you need different proxies per scheme or per host, or want to bypass the proxy for some destinations by mounting a transport with no proxy.
Why does my https:// mount fail when the proxy URL starts with https://?
The scheme in the proxy URL describes how httpx talks to the proxy, not the target. For https:// targets httpx opens a CONNECT tunnel over a plain HTTP connection to the proxy, so the proxy URL should normally use http://. Using https:// asks httpx to negotiate TLS with the proxy itself, which most proxy endpoints do not serve.
Does httpx support SOCKS5 proxies?
Yes, but only after installing the optional dependency with pip install httpx[socks], which pulls in socksio. Then pass proxy="socks5://user:pass@host:port" exactly as you would an HTTP proxy. With Geonode the SOCKS5 rotating port is 11000 and the SOCKS5 sticky port is 12000; HTTP(S) ports do not accept SOCKS handshakes.
Can httpx retry a request automatically when the proxy connection fails?
At the transport level only. httpx.HTTPTransport(retries=2) retries failed connection attempts, including connection errors to the proxy, but it never retries a request that already got a response such as 407 or 502. Retries on HTTP status codes belong in your own loop, where you can switch session id or back off deliberately.
Are there limits to what an httpx proxy setup can do with residential IPs?
A proxy changes your exit IP; it does not change what a site allows. Respect robots.txt, terms of service and rate limits, keep concurrency modest, and avoid collecting personal data you have no basis to process. Residential sessions are also capped at 24 hours, after which the IP is released and you must open a fresh session.
