Node.js HTTP request through a proxy: axios, got and undici, three clients and three mechanics

Three HTTP clients cover most of the Node code I maintain. axios sits in the older services, got runs the collection scripts, and undici carries everything written since fetch arrived inside the runtime. All three send requests through a proxy. They do it in three unrelated ways, and the differences surface at the worst moment, when a job that ran fine for weeks starts printing socket errors against a new target.

The confusion has one root. A proxy for a plain http:// target and a proxy for an https:// target are two mechanisms sharing one word. The first rewrites the request line into absolute form and posts it to the proxy, which reads the whole thing and forwards it. The second opens a CONNECT tunnel and runs TLS with the target inside that tunnel, with the proxy shuffling bytes it cannot read. Every quirk below follows from that split.

I take the clients one at a time, in the order they show up in a codebase, and each section stays with one client from its proxy wiring through retries, timeouts and its socket pool. Numbers in the text came off my own runs against a private server pool where the exit address changes on its own between calls.

Three clients, three ways of reaching the endpoint

Here is the shape of the whole article in one table. Every cell gets its own section below.

The last row decides more arguments than the first nine. got has been ESM only for several major versions, so a CommonJS service either moves to import or stays on an old release. undici needs no install at all, since the runtime already carries it. axios keeps both module formats and a familiar surface, which is why it survives in code nobody wants to touch.

axios: the proxy field and the gap it leaves on HTTPS

The proxy field looks like the answer and covers half the job.

const axios = require('axios');

const res = await axios.get('http://api.example.com/v1/items', {
  proxy: {
    protocol: 'http',
    host: '203.0.113.24',
    port: 8000,
    auth: { username: 'LOGIN', password: 'PASS' },
  },
  timeout: 15000,
});

Against that http:// target the field works exactly as advertised. The Node adapter points the socket at 203.0.113.24:8000, writes the full URL into the request line, attaches a Proxy-Authorization header built from auth, and the proxy forwards the request as a normal HTTP relay.

Point the same configuration at https://api.example.com and the mechanism runs out of road. A tunnel has to be requested with a CONNECT verb before any TLS bytes move, and the proxy field never sends one. What happens next depends on the endpoint. Some answer with an HTTP error page, which arrives at a client waiting for a TLS ServerHello and produces a protocol error. Some drop the socket, which surfaces as socket hang up. The worst case is the quiet one, where a misconfigured job falls back to the host connection and every request leaves through the address of the machine while the log fills with 200s.

Two more behaviours of the field deserve a line each. When proxy is absent, axios reads HTTP_PROXY, HTTPS_PROXY and NO_PROXY from the environment, so a variable exported months ago in a shell profile quietly becomes part of the run. Setting proxy: false shuts both the field and the environment out, which is the state I want whenever an agent takes over the routing.

axios with agents: https-proxy-agent, socks-proxy-agent and the scheme rules

An agent replaces the socket factory below the client, so the tunnel is negotiated before the client sees anything. Two packages cover the HTTP family.

const axios = require('axios');
const { HttpProxyAgent } = require('http-proxy-agent');
const { HttpsProxyAgent } = require('https-proxy-agent');

const PROXY = 'http://LOGIN:PASS@203.0.113.24:8000';

const client = axios.create({
  proxy: false,                       // stop the field and the env from interfering
  httpAgent:  new HttpProxyAgent(PROXY,  { keepAlive: true, maxSockets: 32 }),
  httpsAgent: new HttpsProxyAgent(PROXY, { keepAlive: true, maxSockets: 32 }),
  timeout: 20000,
  headers: { 'Accept-Encoding': 'gzip, deflate' },
});

proxy: false on line one carries real weight. With an agent mounted and the field left at its default, axios can rewrite the request line as well, and the proxy then receives an absolute URL inside a tunnel that was already pointed at the target. The target answers 400 and the message tells you nothing about the cause.

Both keys have to be filled even when the job only touches HTTPS. A single redirect from http:// to https:// swaps the transport mid chain, and an empty httpAgent means that one hop leaves through the host connection.

SOCKS needs one package for both keys, since the agent handles TLS for HTTPS targets by itself.

const { SocksProxyAgent } = require('socks-proxy-agent');

const agent = new SocksProxyAgent('socks5h://LOGIN:PASS@203.0.113.24:1080', {
  keepAlive: true,
  maxSockets: 32,
});

const client = axios.create({ proxy: false, httpAgent: agent, httpsAgent: agent });

The trailing h in the scheme is the detail that decides where hostnames get resolved, and the four schemes differ in ways worth keeping in one place.

I keep socks5h in every script. A lookup performed locally hands the resolver on my machine a list of every hostname the run touches, and it also breaks any target whose DNS answer differs by region of the caller. The same credential pair reaches me as four fields from the panel, IP:PORT:LOGIN:PASS, which converts into the URI above by hand once and by a helper after that. Both the tunnel form and the SOCKS form come off the SOCKS5 pool my Node jobs point at, issued as a list I paste straight into a config file.

axios retries and the two clocks it does not give you

axios has one timeout value and no retry logic. Both gaps are fixable, and both are worth fixing before a run reaches five figures of requests.

The timeout option measures the wait for a response, counted from the moment the request is sent. It does not cap the TCP connect, it does not cap the TLS handshake with the target inside a tunnel, and it does not cap the body download. A hung CONNECT therefore sits outside the number you set. Two additions close that.

const { setTimeout: delay } = require('node:timers/promises');

async function fetchWithRetry(url, tries = 4) {
  for (let attempt = 0; attempt < tries; attempt++) {
    try {
      return await client.get(url, {
        timeout: 20000,
        signal: AbortSignal.timeout(45000),   // wall clock ceiling for the whole call
        validateStatus: (s) => s < 500 && s !== 429,
      });
    } catch (err) {
      const code = err.code || err.response?.status;
      const retryable = ['ECONNRESET', 'ECONNABORTED', 'ETIMEDOUT', 'EPIPE', 'EAI_AGAIN']
        .includes(code) || [408, 429, 500, 502, 503, 504].includes(err.response?.status);
      if (!retryable || attempt === tries - 1) throw err;
      const after = Number(err.response?.headers['retry-after']) || 0;
      await delay(after ? after * 1000 : 600 * 2 ** attempt);
    }
  }
}

AbortSignal.timeout is the outer clock, and it is the only one that covers a socket stuck during the handshake. The agent also accepts its own timeout, which fires on socket inactivity and destroys the connection underneath, which is how I keep a dead tunnel from holding a slot in maxSockets for minutes.

Retry-After deserves the explicit read shown above. A target that sends the header is naming the number of seconds it wants, and honouring that number keeps a temporary throttle from turning into a long one. The doubling series behind it runs 0.6 s, 1.2 s, 2.4 s, 4.8 s, which is gentle enough for a rate limit and short enough to survive a rotation event.

The axios-retry package does the same work with less code and adds one thing my loop lacks: it keeps the interceptor chain intact, so a request rebuilt for a second attempt still passes through anything that signs or logs it. What it cannot do is make POST safe. My retryable list stays on GET and HEAD unless the endpoint documents an idempotency key.

got: agents keyed by the protocol of the target

got has no proxy option and never will. The maintainers pushed the job down to the agent layer, and the shape that comes out of that decision is the most predictable of the three.

import got from 'got';
import { HttpProxyAgent } from 'http-proxy-agent';
import { HttpsProxyAgent } from 'https-proxy-agent';

const PROXY = 'http://LOGIN:PASS@203.0.113.24:8000';

const client = got.extend({
  agent: {
    http:  new HttpProxyAgent(PROXY,  { keepAlive: true, maxSockets: 24 }),
    https: new HttpsProxyAgent(PROXY, { keepAlive: true, maxSockets: 24 }),
  },
  headers: { 'user-agent': 'collector/1.4' },
  http2: false,
  throwHttpErrors: false,
});

Read the keys the right way around and the whole thing stops being confusing. http and https name the protocol of the target URL. The agent under each key carries the transport toward the proxy. Two different schemes living inside one small object, and mixing them up produces a client that works on one target and fails on the next.

got reads nothing from the environment. No HTTP_PROXY, no NO_PROXY, no registry lookup on Windows. Some people call that an omission. I call it the reason got scripts behave identically on my laptop and on a runner, and the reason I never spend an evening chasing a variable somebody exported in a Dockerfile.

Two more notes from production. Setting http2: true while an agent is mounted drops the agent, since the HTTP/2 path uses a different pool, so a proxied got client stays on HTTP/1.1. And throwHttpErrors: false turns 4xx and 5xx into ordinary responses, which lets my code read the body of a refusal, log the headers that came with it, and decide what to do with the URL.

got retry and the seven timeout phases

This is where got earns its place. The retry object handles both error codes and status codes, and the timeout object splits a request into phases that can each be capped on their own.

const client = got.extend({
  agent: { http: httpAgent, https: httpsAgent },
  retry: {
    limit: 3,
    methods: ['GET', 'HEAD'],
    statusCodes: [408, 413, 429, 500, 502, 503, 504, 521, 522, 524],
    errorCodes: ['ETIMEDOUT', 'ECONNRESET', 'ECONNREFUSED', 'EPIPE', 'EAI_AGAIN', 'EPROTO'],
    calculateDelay: ({ attemptCount, error, computedValue }) => {
      if (error.response?.statusCode === 407) return 0;   // stop, credentials will not improve
      return Math.min(computedValue, 8000);
    },
  },
  timeout: {
    lookup: 500,
    connect: 4000,
    secureConnect: 4000,
    socket: 12000,
    send: 10000,
    response: 20000,
    request: 45000,
  },
  hooks: {
    beforeRetry: [(error, count) => console.warn('retry', count, error.code, error.request?.requestUrl)],
  },
});

Three phases behave differently once a proxy sits in the path, and knowing which is which saves an hour of guessing.

lookup covers DNS for the proxy host, since that is the only name the Node process resolves when a tunnel is in use. On a numeric endpoint the phase never fires at all. connect covers the TCP handshake with the proxy plus the CONNECT exchange. secureConnect covers the TLS handshake with the target, negotiated inside the tunnel, which is why a slow target shows up here and a slow proxy shows up one phase earlier. response measures the wait for the first byte of the answer, and request caps everything together including the body.

calculateDelay returning 0 stops the sequence, and 407 belongs in that branch. A Proxy Authentication Required on one worker means the same answer on all of them, so a repeat spends the whole retry budget confirming what the first attempt already said. Three causes cover nearly every one I have seen: a pasted trailing space in the password, a machine whose address was never bound in the panel, or an access period that ended overnight. Binding takes seconds, the package carries two bindable addresses that swap freely, and jobs on runners with a changing address use the login form. Both connection styles sit in the same dashboard, and the login form arrives with HTTP endpoints that answer a credential pair.

undici: ProxyAgent, the global dispatcher and the built-in fetch

undici replaces agents with dispatchers, and ProxyAgent is a dispatcher that owns the tunnel.

import { ProxyAgent, setGlobalDispatcher, request } from 'undici';

const dispatcher = new ProxyAgent({
  uri: 'http://203.0.113.24:8000',
  token: 'Basic ' + Buffer.from('LOGIN:PASS').toString('base64'),
  connections: 32,
  headersTimeout: 15000,
  bodyTimeout: 30000,
  keepAliveTimeout: 10000,
  keepAliveMaxTimeout: 120000,
  connect: { timeout: 5000 },
});

const { statusCode, headers, body } = await request('https://api.example.com/items', { dispatcher });
const data = await body.json();

One dispatcher handles both protocols. An https:// target gets a CONNECT tunnel with TLS negotiated end to end through it. An http:// target gets the absolute request form written for it. Credentials travel either inside the uri or through token, and the header form is the one I prefer, since a password inside a URI ends up in every log line that prints the dispatcher.

Two options in that block cover cases nothing else covers. requestTls sets the TLS parameters used with the target inside the tunnel, including SNI. proxyTls sets the parameters for TLS toward the proxy itself, which only matters on an https:// endpoint URI.

The global form is what makes the runtime's own fetch follow the same path:

setGlobalDispatcher(dispatcher);
const res = await fetch('https://api.ipify.org?format=json');   // now proxied

Per call routing works too, since fetch in Node accepts a non standard dispatcher option, and I use that form in anything where one target has to stay local:

const res = await fetch(url, { dispatcher });

Environment variables have their own dispatcher. EnvHttpProxyAgent reads HTTP_PROXY, HTTPS_PROXY and NO_PROXY at construction and routes each request by matching the hostname against the no-proxy list, which is the closest undici comes to the behaviour axios has by default. Newer runtimes also carry a flag that switches the built-in fetch to that agent without any code.

SOCKS has no first party support here, and a custom connector fills the gap in twelve lines:

import { Agent } from 'undici';
import { SocksClient } from 'socks';

const socksDispatcher = new Agent({
  connections: 32,
  connect: async ({ hostname, port, protocol }, callback) => {
    try {
      const { socket } = await SocksClient.createConnection({
        proxy: { host: '203.0.113.24', port: 1080, type: 5, userId: 'LOGIN', password: 'PASS' },
        command: 'connect',
        destination: { host: hostname, port: Number(port) || (protocol === 'https:' ? 443 : 80) },
      });
      callback(null, socket);
    } catch (err) {
      callback(err, null);
    }
  },
});

That connector hands undici a socket that already sits on the far side of the SOCKS handshake, with the hostname resolved by the proxy because the destination went over as a name. The endpoints I feed it come from a SOCKS5 endpoint list I paste into config, the same list the axios agent above reads.

One habit belongs to undici.request alone: the body must be consumed. A response whose body is never read holds its socket out of the pool until a timeout releases it, and a loop that checks statusCode and moves on will starve itself within a few hundred iterations. await body.dump() on any path that skips the payload is the entire fix.

undici retries, pool size and keep-alive under a thread ceiling

Retries wrap the dispatcher, which keeps the logic out of the call site.

import { RetryAgent, ProxyAgent, setGlobalDispatcher } from 'undici';

const proxy = new ProxyAgent({ uri: 'http://203.0.113.24:8000', connections: 32 });

setGlobalDispatcher(new RetryAgent(proxy, {
  maxRetries: 3,
  minTimeout: 600,
  maxTimeout: 8000,
  timeoutFactor: 2,
  retryAfter: true,
  methods: ['GET', 'HEAD'],
  statusCodes: [429, 500, 502, 503, 504],
  errorCodes: ['ECONNRESET', 'ECONNREFUSED', 'ETIMEDOUT', 'EPIPE', 'UND_ERR_SOCKET'],
}));

connections is the socket ceiling per origin, and through a tunnel each target origin gets its own set of tunnels. Twelve target hosts at 32 connections each means up to 384 parallel tunnels toward the proxy, which is the number your package has to allow, and the number people forget when they raise concurrency on a multi host job.

The thread arithmetic is short and I run it before every new job. The packages I use permit 1000 parallel connections, the corporate term goes to 3000, and they do not stack, so two packages on one account still leave the higher figure as the ceiling. Binding two addresses splits that allowance in half across them. Whatever the client, the working number is concurrent sockets, and traffic volume never enters it, since an access term with the transfer meter switched off leaves concurrency as the only quantity to plan around.

Keep-alive settings decide how much of that ceiling you actually use. keepAliveTimeout is how long an idle socket survives, with a default measured in seconds. keepAliveMaxTimeout caps what a server's Keep-Alive header can extend it to. Raise the first on a job that hits the same host all night and the handshake cost disappears from the timings. Leave it low on a wide sweep across hundreds of hosts and the pool stops hoarding sockets it will never reuse.

The equivalent knob in axios and got is maxSockets on the Node agent, defaulting to Infinity, which is the least useful default in the runtime. An uncapped agent under 200 concurrent tasks opens 200 tunnels, hits the package ceiling, and the errors that come back read like network faults. I size it at the worker count plus a small margin and let the queue form inside the agent, where I can see it. Pool sizing against target depth is the same exercise for all three clients, and the pages describing pool access taken by the month list the parallel connection figure that each access term allows.

Checking the exit address from all three clients

Every run I start asks what the world sees before it touches the target. One request, three lines, and it has caught more misconfiguration for me than any log parsing.

// axios
const a = await client.get('https://api.ipify.org').then(r => r.data.trim());

// got
const g = await client('https://api.ipify.org').then(r => r.body.trim());

// undici
const { body } = await request('https://api.ipify.org', { dispatcher });
const u = (await body.text()).trim();

const direct = await fetch('https://api.ipify.org', { dispatcher: undefined })
  .then(r => r.text()).then(t => t.trim());

if (a === direct) throw new Error('axios traffic is leaving through the host address');

The thrown error is the part that matters. A printed address proves nothing at three in the morning; a failing assertion stops the run before it burns through a worker budget.

During a run I sample the same call from a random worker every few hundred requests and write the answer into a counter. Two readings come out of it. The spread tells me how many distinct exits the batch touched, which on a pool of roughly 12 000 active addresses across more than 200 countries widens quickly. The repeat rate tells me whether one address is carrying more of the batch than the rest, which is a signal to raise the worker count so the pool has more reason to hand out fresh sockets. Both readings behave the way they do because the exits sit on IPv4 addresses standing on server hardware with rotation performed inside the pool between calls.

Check the protocol as well as the address. A request pointed at an HTTP endpoint through an agent built for SOCKS will sometimes succeed and behave oddly under load, and comparing what a header reflection endpoint reports against what you sent takes ten seconds.

Node error messages, line by line

Node reports proxy trouble through error codes that mostly predate proxies. Here is the table I keep open while debugging, with the reading that has proven right most often.

Four of those rows carry most of the traffic in my logs, and each one deserves the longer reading.

ECONNRESET means the remote end sent a TCP reset. On a rotating pool a low steady rate of these is normal operation, because a socket that was fine a moment ago belongs to an exit that has since moved on. One retry recovers it. The number worth watching is the rate per hour: flat is healthy, and a rate climbing through the night says the worker count outgrew what the target tolerates. A sudden spike concentrated on one host says that host started dropping your traffic deliberately.

socket hang up is what the Node HTTP parser prints when the connection closes before a response line arrives. Through a proxy it has three regular causes. The endpoint port speaks SOCKS while the client sends HTTP, or the reverse. The CONNECT was refused and the proxy hung up without writing a status. Or a keep-alive socket was reused microseconds after the far side decided to close it, which is a race that gets rarer as keepAliveTimeout drops below the server's own idle window.

EPROTO comes out of OpenSSL and almost always means TLS bytes reached something that was speaking plain text. In proxy code it has one dominant cause: an https:// scheme in the proxy URI pointed at a port that terminates plain HTTP. The scheme of the proxy URI describes the transport to the proxy, and the target scheme is a separate matter carried by the tunnel. Write http://LOGIN:PASS@host:8000 for a plain endpoint even when every request in the job goes to an HTTPS target.

ERR_TLS_CERT_ALTNAME_INVALID tells you the certificate you got belongs to some other hostname. Inside a tunnel the usual reason is a missing or wrong SNI, which happens when a client is told to connect to an address while the target expects a name. Set servername on the TLS options and it resolves. What I never do is switch verification off. A run with rejectUnauthorized: false in it accepts whatever certificate arrives, and the day it accepts one from an interception layer is the day the credentials in your headers stop being yours.

Keep a small log of error codes by hour with the exit address attached. Four columns, plain text, written by the same handler that counts retries. After two runs the pattern reads itself, and the difference between a target tightening up and an endpoint going quiet becomes obvious at a glance.

The same wiring in other stacks is covered by the neighbouring guides: every option above expressed as command line arguments sits in the curl reference for proxied calls, the Python equivalent of sessions, pools and retry objects is in the requests and urllib3 guide, and the same three way split between plain HTTP, tunnels and SOCKS appears in the PHP notes on cURL and Guzzle. Before pasting an endpoint into any of the code above, run the four field form through the connection string parser on this site, which turns IP:PORT:LOGIN:PASS into the URI shape each of these three clients expects.