C# HttpClient Proxy Setup: Credentials, Timeouts | Geonode

Q: How do I set an HttpClient proxy with a username and password in C#?

Create a WebProxy with the proxy address and set its Credentials property to a NetworkCredential holding the username and password. Assign that WebProxy to the Proxy property of an HttpClientHandler or SocketsHttpHandler, set UseProxy to true, and pass the handler to the HttpClient constructor. The handler sends the credentials when the proxy answers 407.

Q: Should I use HttpClientHandler or SocketsHttpHandler for a proxy?

On .NET Core 2.1 and later HttpClientHandler is a thin wrapper over SocketsHttpHandler, so both work. SocketsHttpHandler exposes PooledConnectionLifetime, which lets you recycle pooled connections periodically. That matters with rotating proxies, because a reused connection keeps the same exit IP until it closes. Prefer SocketsHttpHandler for new code.

Q: Why does my HttpClient proxy request throw TaskCanceledException?

HttpClient reports a timeout as TaskCanceledException, not as a dedicated timeout exception. It is thrown when HttpClient.Timeout elapses or when the CancellationToken you passed is cancelled. On .NET 5 and later the inner exception is a TimeoutException when the client timeout was the cause, which lets you tell the two apart.

Q: Does HttpClient read HTTP_PROXY and HTTPS_PROXY environment variables?

Yes. On .NET Core 3.0 and later, HttpClient.DefaultProxy is initialised from HTTP_PROXY, HTTPS_PROXY and NO_PROXY, and handlers use it when no explicit Proxy is set. Credentials can be included in the URL. Setting the environment variables is convenient in containers, but an explicit WebProxy in code is easier to reason about.

This guide shows how to configure an HttpClient proxy in C# on .NET 8: a WebProxy with NetworkCredential, the choice between HttpClientHandler and SocketsHttpHandler, correct client reuse with IHttpClientFactory, timeouts and cancellation, environment-variable configuration, bypass lists, and the two exceptions you will actually meet in production. It ends with a complete console program that opens a sticky residential session, verifies the exit IP and retries with a new session on failure.

Everything below targets .NET 8 with top-level statements, but the API has been stable since .NET Core 3.0 and the same code compiles on .NET 6 and 7.

WebProxy and NetworkCredential

The building block is System.Net.WebProxy, documented on learn.microsoft.com. It holds the proxy address and, through its Credentials property, the username and password the handler sends when the proxy challenges with 407.

using System.Net;

var proxy = new WebProxy("http://premium-residential.geonode.com:9000")
{
    Credentials = new NetworkCredential("USERNAME", "PASSWORD")
};

The address scheme is http:// even when the targets are HTTPS. For an https:// URL the handler sends a CONNECT request to the proxy and runs TLS with the target inside that tunnel, so the proxy hop itself stays plain HTTP and certificate validation applies to the target exactly as it would without a proxy.

Because the credentials live in a NetworkCredential rather than in the URL, you do not need to percent-encode special characters in the password. That is one practical advantage over clients that only accept userinfo in the proxy URL.

HttpClientHandler versus SocketsHttpHandler

Two handler classes accept a proxy. HttpClientHandler is the long-standing API; since .NET Core 2.1 it delegates to SocketsHttpHandler on all platforms, so there is no functional difference for proxy support. The reason to use SocketsHttpHandler directly is the extra knobs it exposes, and one of them matters a great deal for proxies.

// Option A: HttpClientHandler
var handlerA = new HttpClientHandler
{
    Proxy = proxy,
    UseProxy = true
};

// Option B: SocketsHttpHandler (preferred for new code)
var handlerB = new SocketsHttpHandler
{
    Proxy = proxy,
    UseProxy = true,
    PooledConnectionLifetime = TimeSpan.FromMinutes(2)
};

PooledConnectionLifetime is the knob. HttpClient pools connections and reuses them across requests. With a rotating proxy on port 9000, the exit IP is chosen when a connection to the proxy is established; subsequent requests on the same pooled connection may keep the same exit. Giving pooled connections a bounded lifetime forces periodic reconnection, which also picks up DNS changes for the proxy host. Two minutes is a reasonable default; shorten it if you want rotation to happen more often than your connection churn naturally provides.

The table summarises when each option fits.

Need	Handler	Setting
Minimal proxy wiring, existing code	`HttpClientHandler`	`Proxy`, `UseProxy = true`
Bounded connection reuse with rotating IPs	`SocketsHttpHandler`	`PooledConnectionLifetime`
Limit connections per proxy host	`SocketsHttpHandler`	`MaxConnectionsPerServer`
Disable proxy for one client	either	`UseProxy = false`

Reuse one HttpClient

HttpClient is designed to be created once and reused for the lifetime of the application. Creating a new instance per request exhausts sockets under load, and with a proxy it also throws away the pooled CONNECT tunnels you just paid latency to establish. In a console app or worker, hold a single static instance.

var http = new HttpClient(handlerB)
{
    Timeout = TimeSpan.FromSeconds(30)
};

IHttpClientFactory in ASP.NET Core

In ASP.NET Core, register a named client and attach the handler through ConfigurePrimaryHttpMessageHandler. The factory manages handler lifetime for you and rotates handlers on a schedule, which avoids both the socket exhaustion problem and stale DNS.

// Program.cs
using System.Net;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHttpClient("geonode", client =>
{
    client.Timeout = TimeSpan.FromSeconds(30);
})
.ConfigurePrimaryHttpMessageHandler(() =>
{
    var proxy = new WebProxy("http://premium-residential.geonode.com:9000")
    {
        Credentials = new NetworkCredential(
            Environment.GetEnvironmentVariable("GEONODE_USER"),
            Environment.GetEnvironmentVariable("GEONODE_PASS"))
    };

    return new SocketsHttpHandler
    {
        Proxy = proxy,
        UseProxy = true,
        PooledConnectionLifetime = TimeSpan.FromMinutes(2)
    };
});

var app = builder.Build();

app.MapGet("/ip", async (IHttpClientFactory factory) =>
{
    var client = factory.CreateClient("geonode");
    return await client.GetStringAsync("https://api.ipify.org?format=json");
});

app.Run();

Inject IHttpClientFactory wherever you need the client and call CreateClient("geonode"). The returned HttpClient is cheap to create because the expensive handler underneath is shared.

Timeouts and CancellationToken

HttpClient.Timeout is the total time allowed for a request, including connecting to the proxy, the CONNECT handshake, TLS with the target and reading the response headers. The default is 100 seconds, which is too long for a scraper worker. Set it on the client and, for finer control, pass a CancellationToken per request.

using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15));

try
{
    var body = await http.GetStringAsync("https://example.com/", cts.Token);
}
catch (TaskCanceledException ex) when (ex.InnerException is TimeoutException)
{
    // HttpClient.Timeout elapsed
}
catch (TaskCanceledException)
{
    // the CancellationToken was cancelled (our 15 s budget)
}

Residential exits add latency compared to a direct connection, so budget a few extra seconds per request. A consistent timeout at the same stage across many requests usually indicates a network path problem to the proxy host, not a slow target.

DefaultProxy and environment variables

If you set no Proxy on the handler, .NET uses HttpClient.DefaultProxy. On .NET Core 3.0 and later that static property is initialised from the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables, and the URL may contain credentials.

export HTTP_PROXY="http://USERNAME:PASSWORD@premium-residential.geonode.com:9000"
export HTTPS_PROXY="http://USERNAME:PASSWORD@premium-residential.geonode.com:9000"
export NO_PROXY="localhost,127.0.0.1,.internal"

You can also assign HttpClient.DefaultProxy in code at startup, which affects every handler that does not override Proxy. This is handy in containers, but it is global state: a health-check client or a call to an internal API will go through the residential proxy too unless NO_PROXY lists those hosts. For anything beyond a throwaway script, an explicit WebProxy on a named handler is clearer.

BypassList and BypassProxyOnLocal

WebProxy has its own exclusion mechanism. BypassProxyOnLocal = true skips the proxy for hosts without a dot in the name (such as localhost or a bare machine name). BypassList takes regular expressions matched against the target URI.

var proxy = new WebProxy("http://premium-residential.geonode.com:9000")
{
    Credentials = new NetworkCredential("USERNAME", "PASSWORD"),
    BypassProxyOnLocal = true,
    BypassList = new[] { @"^https?://(.*\.)?internal\.example(:\d+)?/" }
};

Remember that these are regex patterns, not glob patterns; an unescaped dot matches any character.

Handling 407 and other failures

Two failure modes account for nearly every HttpClient proxy ticket. The first is a 407 Proxy Authentication Required, which means the proxy rejected the credentials before forwarding anything. It usually surfaces as an HttpRequestException, and depending on the .NET version and the point of failure you may instead get a response whose status code is HttpStatusCode.ProxyAuthenticationRequired. Check both.

using System.Net;
using System.Net.Http;

try
{
    using var response = await http.GetAsync("https://example.com/");

    if (response.StatusCode == HttpStatusCode.ProxyAuthenticationRequired)
    {
        throw new InvalidOperationException("proxy rejected credentials (407)");
    }

    response.EnsureSuccessStatusCode();
    var html = await response.Content.ReadAsStringAsync();
}
catch (HttpRequestException ex) when (ex.StatusCode == HttpStatusCode.ProxyAuthenticationRequired)
{
    // Also 407: fix the username grammar or password; retrying will not help.
}
catch (HttpRequestException ex)
{
    // Connection refused, DNS failure, TLS error, or EnsureSuccessStatusCode on 4xx/5xx.
    Console.Error.WriteLine($"request failed: {ex.Message} (status {ex.StatusCode})");
}

The second is TaskCanceledException for timeouts, covered above. The important rule for both: a 407 is a configuration error and must not be retried; a timeout or a 5xx from the target is transient and is a reasonable candidate for a retry on a new session.

Sticky sessions through the username

Port 9000 gives a new exit IP for every new connection. When a flow needs one IP for several requests (login followed by data pages, or paginated results), use port 10000 and select the session through the username. The grammar appends suffixes to your base username: -country-xx for geo targeting, -session-XXXXXXXX with an 8-character alphanumeric id, and optionally -lifetime-SECONDS from 180 up to 86400.

static string NewSessionId() =>
    Guid.NewGuid().ToString("N")[..8]; // 8 alphanumeric chars

static WebProxy StickyProxy(string baseUser, string pass, string country = "us", int lifetime = 600)
{
    var user = $"{baseUser}-country-{country}-session-{NewSessionId()}-lifetime-{lifetime}";
    return new WebProxy("http://premium-residential.geonode.com:10000")
    {
        Credentials = new NetworkCredential(user, pass)
    };
}

Every request through a client built on this proxy shares the same exit IP for ten minutes. To switch IP, build a new handler with a new session id. Since the session is bound to the credentials, two clients with the same username share the same exit even across processes.

Verifying the exit IP

Never assume the HttpClient proxy is in use; confirm it. Request https://api.ipify.org?format=json (or https://httpbin.org/ip) through the client and compare with your real address. With a sticky session two calls return the same IP; with port 9000 and a fresh connection they differ.

using System.Text.Json;

var json = await http.GetStringAsync("https://api.ipify.org?format=json");
var ip = JsonDocument.Parse(json).RootElement.GetProperty("ip").GetString();
Console.WriteLine($"exit IP: {ip}");

Full console example (.NET 8)

The program below ties everything together: a sticky session, bounded timeouts, 407 detection, and up to three attempts with a new session on transient failures. Create it with dotnet new console -n ProxyDemo, replace Program.cs, set GEONODE_USER and GEONODE_PASS, and run dotnet run.

// Program.cs  (.NET 8, top-level statements)
using System.Net;
using System.Text.Json;

var baseUser = Environment.GetEnvironmentVariable("GEONODE_USER")
    ?? throw new InvalidOperationException("GEONODE_USER not set");
var password = Environment.GetEnvironmentVariable("GEONODE_PASS")
    ?? throw new InvalidOperationException("GEONODE_PASS not set");

var target = args.Length > 0 ? args[0] : "https://api.ipify.org?format=json";

for (var attempt = 1; attempt <= 3; attempt++)
{
    var sessionId = Guid.NewGuid().ToString("N")[..8];
    var user = $"{baseUser}-country-us-session-{sessionId}-lifetime-600";

    var handler = new SocketsHttpHandler
    {
        Proxy = new WebProxy("http://premium-residential.geonode.com:10000")
        {
            Credentials = new NetworkCredential(user, password)
        },
        UseProxy = true,
        PooledConnectionLifetime = TimeSpan.FromMinutes(2)
    };

    using var http = new HttpClient(handler) { Timeout = TimeSpan.FromSeconds(30) };
    using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(20));

    try
    {
        // 1. Confirm the exit IP for this session.
        var ipJson = await http.GetStringAsync("https://api.ipify.org?format=json", cts.Token);
        var ip = JsonDocument.Parse(ipJson).RootElement.GetProperty("ip").GetString();
        Console.WriteLine($"[attempt {attempt}] session {sessionId} exit IP {ip}");

        // 2. Fetch the real target on the same IP.
        using var response = await http.GetAsync(target, cts.Token);

        if (response.StatusCode == HttpStatusCode.ProxyAuthenticationRequired)
        {
            Console.Error.WriteLine("407: proxy rejected credentials, not retrying");
            return;
        }

        if ((int)response.StatusCode is 403 or 429 or >= 500)
        {
            Console.Error.WriteLine($"[attempt {attempt}] got {(int)response.StatusCode}, rotating session");
            continue;
        }

        var body = await response.Content.ReadAsStringAsync(cts.Token);
        Console.WriteLine($"{(int)response.StatusCode} {target} ({body.Length} chars)");
        return;
    }
    catch (HttpRequestException ex) when (ex.StatusCode == HttpStatusCode.ProxyAuthenticationRequired)
    {
        Console.Error.WriteLine("407: proxy rejected credentials, not retrying");
        return;
    }
    catch (HttpRequestException ex)
    {
        Console.Error.WriteLine($"[attempt {attempt}] network error: {ex.Message}");
    }
    catch (TaskCanceledException)
    {
        Console.Error.WriteLine($"[attempt {attempt}] timed out");
    }

    await Task.Delay(TimeSpan.FromMilliseconds(500 * attempt));
}

Console.Error.WriteLine("gave up after 3 attempts");

The handler is recreated per attempt on purpose: a sticky session is tied to the username in the credentials, so a new session requires a new WebProxy. In a long-running service you would cache one client per session id rather than per request.

Common mistakes

Creating an HttpClient per request

Each new client opens a new CONNECT tunnel and leaves sockets in TIME_WAIT. Reuse one instance, or use IHttpClientFactory, and let PooledConnectionLifetime handle rotation.

Using an https:// proxy address

The proxy hop is plain HTTP; HTTPS targets travel inside the CONNECT tunnel. An https:// scheme on the WebProxy address tells .NET to negotiate TLS with the proxy itself, which fails against an endpoint that expects plain HTTP.

Retrying a 407

A 407 means the proxy did not accept the credentials. Retrying with the same username and password only wastes time. Fail fast and check the username grammar, especially the session id length and the lifetime range.

Ignoring NO_PROXY and BypassList

When the proxy comes from DefaultProxy, every outbound call in the process uses it, including health checks and internal APIs. List those hosts in NO_PROXY or BypassList, or configure the proxy on a named client only.

Next step

The code above runs unchanged against Geonode residential proxies: 2.5M IPs online across 195+ countries, HTTP, HTTPS and SOCKS5 on one host, and sticky sessions up to 24 hours selected through the username grammar shown here. Country, state, city and ASN targeting use the same suffixes, so narrowing the exit location is a string change, not a code change. Pricing is pay-as-you-go with no contract, from 10 GB at $7.92 per month, with a 3-day trial for $5. See residential proxies for details, or compare tiers on the pricing page. For the rest of the pipeline, start with the web scraping use case.

FAQ

How do I set an HttpClient proxy with a username and password in C#?

Create a WebProxy with the proxy address and set its Credentials property to a NetworkCredential holding the username and password. Assign that WebProxy to the Proxy property of an HttpClientHandler or SocketsHttpHandler, set UseProxy to true, and pass the handler to the HttpClient constructor. The handler sends the credentials when the proxy answers 407.

Should I use HttpClientHandler or SocketsHttpHandler for a proxy?

On .NET Core 2.1 and later HttpClientHandler is a thin wrapper over SocketsHttpHandler, so both work. SocketsHttpHandler exposes PooledConnectionLifetime, which lets you recycle pooled connections periodically. That matters with rotating proxies, because a reused connection keeps the same exit IP until it closes. Prefer SocketsHttpHandler for new code.

Why does my HttpClient proxy request throw TaskCanceledException?

HttpClient reports a timeout as TaskCanceledException, not as a dedicated timeout exception. It is thrown when HttpClient.Timeout elapses or when the CancellationToken you passed is cancelled. On .NET 5 and later the inner exception is a TimeoutException when the client timeout was the cause, which lets you tell the two apart.

Does HttpClient read HTTP_PROXY and HTTPS_PROXY environment variables?

Yes. On .NET Core 3.0 and later, HttpClient.DefaultProxy is initialised from HTTP_PROXY, HTTPS_PROXY and NO_PROXY, and handlers use it when no explicit Proxy is set. Credentials can be included in the URL. Setting the environment variables is convenient in containers, but an explicit WebProxy in code is easier to reason about.

How do I keep the same IP across several HttpClient proxy requests?

Use the sticky port 10000 and add a session suffix to the username, for example USER-country-us-session-a1b2c3d4-lifetime-600. All requests sent with that username share one exit IP for the lifetime given in seconds, between 180 and 86400. Generate a new 8-character session id when you want a fresh IP.

What limits apply when scraping through a residential HttpClient proxy?

Bandwidth is billed per gigabyte, so download only what you need and avoid images where possible. Sticky sessions hold an IP for at most 24 hours and may end earlier if the device goes offline, so your code must handle an IP change. A proxy does not exempt you from a site's terms of service, robots directives or rate limits.

C# HttpClient Proxy Setup: Credentials, Timeouts and Verification