PHP cURL proxy: the options, the SOCKS5 letter and the Guzzle client that reuses them

My collection code in PHP started as a single curl_setopt call and one hard coded endpoint string. It fetched a page, printed the body, and I was satisfied for about a week. Then the same file grew a queue, a parallel window, a retry policy and a second client library on top, and every one of those additions exposed a proxy option I had been setting wrong without noticing.

This guide walks that growth in order. The raw extension first, because Guzzle sits on it and every strange behaviour resolves to a libcurl option underneath. Then the client, the handler, the retry middleware and the error numbers that arrive in your log at four in the morning. The figures come off a job I run nightly: 1400 detail pages across 6 domains, PHP 8 on a small server, endpoints from a pool of private server addresses with rotation inside the pool.

Nothing here needs a framework. A plain php script.php reproduces every block.

CURLOPT_PROXY and the string libcurl accepts

One option carries the whole configuration if you write it fully.

<?php
$ch = curl_init('https://example.net/catalog/page/3');

curl_setopt_array($ch, [
    CURLOPT_PROXY          => 'http://203.0.113.24:8000',
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CONNECTTIMEOUT => 6,
    CURLOPT_TIMEOUT        => 40,
]);

$body = curl_exec($ch);
$code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);

echo $code, ' ', strlen((string) $body), " bytes\n";

The string accepts three shapes and fills in whatever you leave out. A bare 203.0.113.24 gives you an HTTP proxy on port 1080, since 1080 is the assumed port and it belongs to SOCKS. That default cost me two debugging sessions on endpoints that were healthy the whole time. A string with a port and no scheme gives you HTTP on that port. A string with both gives you exactly what you typed, and that is the only form I let into a repository.

CURLOPT_PROXYPORT sets the port separately as an integer. It duplicates information the string already carries, and when the two disagree the reader of your code has to know which wins. I keep the port in the string.

Two neighbours matter early. CURLOPT_HTTPPROXYTUNNEL forces a CONNECT tunnel even for plain HTTP targets, which changes what the endpoint sees from a full request line to a host and a port. CURLOPT_NOPROXY takes a comma separated host list that bypasses the proxy, and any internal service the script also talks to belongs there. A metrics push to your own collector sent through a far endpoint hangs, times out, and produces an error pointing nowhere near the cause.

The dashboard format decides how much parsing you write. My list arrives as IP:PORT for machines I have bound and IP:PORT:LOGIN:PASS for everything else, so the conversion into a CURLOPT_PROXY string is two explode calls. Both shapes come with an HTTP proxy package that carries either connection form, which is why one helper function serves my bound production node and my laptop.

function proxyFromLine(string $line): array
{
    $p = explode(':', trim($line));
    $opts = [CURLOPT_PROXY => 'http://' . $p[0] . ':' . $p[1]];
    if (count($p) === 4) {
        $opts[CURLOPT_PROXYUSERNAME] = $p[2];
        $opts[CURLOPT_PROXYPASSWORD] = $p[3];
    }
    return $opts;
}

CURLOPT_PROXYTYPE, the constants and the letter that moves DNS

CURLOPT_PROXYTYPE takes a constant and declares the transport between your PHP process and the endpoint. The set worth knowing:

CURLPROXY_HTTP is the default and the one you get when you write nothing. CURLPROXY_HTTP_1_0 pins the tunnel negotiation to the older protocol version, which a few ancient gateways still require. CURLPROXY_HTTPS wraps the first hop in TLS and appears only in builds with a recent libcurl, so check curl_version() before shipping it. CURLPROXY_SOCKS4 and CURLPROXY_SOCKS4A are museum pieces. The pair that earns its own paragraphs is CURLPROXY_SOCKS5 and CURLPROXY_SOCKS5_HOSTNAME.

Both speak SOCKS5. They differ on one question: which machine performs the DNS lookup for your target hostname.

With CURLPROXY_SOCKS5, PHP resolves example.net on the machine running the script and hands the resulting IP address to the endpoint. With CURLPROXY_SOCKS5_HOSTNAME, PHP hands over the literal hostname and the exit performs the lookup. The scheme prefixes socks5:// and socks5h:// in the proxy string mean the same two things, and the trailing h stands for the hostname form.

Two consequences follow, and both go beyond syntax.

The first is what the target answers. A hostname that resolves to a regional edge node from your network and to a different origin from the exit produces two different pages, two different header sets and two different cookie policies in the same minute. I watched exactly that on a retail site with a geo aware CDN, and the run looked broken until I compared the two resolutions side by side.

The second is exposure. Local resolution writes every target domain into your own resolver log and into your provider's, so the tunnel carries the request bodies while the list of who you are talking to sits in plain view outside it. For collection work I treat that as settled, and socks5h goes into the config file so nobody has to remember the letter.

$ch = curl_init('https://example.net/api/items');

curl_setopt_array($ch, [
    CURLOPT_PROXY          => '203.0.113.60:1080',
    CURLOPT_PROXYTYPE      => CURLPROXY_SOCKS5_HOSTNAME,
    CURLOPT_PROXYUSERPWD   => 'A19f42:x7Qd21ka',
    CURLOPT_RETURNTRANSFER => true,
]);

Writing a scheme into the string and a constant into CURLOPT_PROXYTYPE at the same time is where people get caught, since two sources of truth for one setting will disagree the day someone edits half of it. Pick one. I use the bare host:port string with an explicit constant in extension code, and the full socks5h:// string in Guzzle. Jobs addressed entirely by hostname run through SOCKS5 endpoints that resolve names at the exit, and the check that proves it took one request.

To prove which side resolved the name, point the same handle at an echo service under both constants and compare. Identical bodies mean the two paths agree. A difference is worth knowing before the endpoint enters a batch.

CURLOPT_PROXYUSERPWD, the auth options and the map to Guzzle

CURLOPT_PROXYUSERPWD takes one string of login:password. It breaks on one input: a password containing a colon. libcurl splits on the first colon, so A19f42:x7Q:d21ka sends the password x7Q and produces a refusal that reads like a wrong account.

CURLOPT_PROXYUSERNAME and CURLOPT_PROXYPASSWORD take the two halves separately and have no such edge. The same reasoning applies to characters like @ and #, which have to be percent encoded when the pair rides inside the proxy URL and can stay raw in the separate options.

CURLOPT_PROXYAUTH picks the scheme. CURLAUTH_BASIC is the default and the one our endpoints speak. CURLAUTH_NTLM and CURLAUTH_DIGEST show up on corporate gateways. CURLAUTH_ANY asks libcurl to read the Proxy-Authenticate header and negotiate, which costs an extra round trip on every request. On a batch of 1400 pages that added 41 seconds of pure waiting before I pinned the value.

The path with no credential at all is address binding. I register the server's own address in the dashboard, and after that the endpoint answers that machine with an empty auth header, so nothing secret sits in the repository or the process table. Packages come with two bindable addresses, enough for a production node and a staging node. Long running jobs on my own hardware sit on bound addresses, while anything running from a shifting address keeps the login form over an HTTPS hop that encrypts the credential exchange.

Here is the map I keep pinned, because switching between the extension and the client twice in one afternoon scrambles the names.

The curl key in Guzzle takes a raw array of these constants and passes them to the handle, so anything missing from the client API stays reachable in one line.

Two timeouts, and the millisecond pair that needs NOSIGNAL

CURLOPT_CONNECTTIMEOUT caps the time to reach the endpoint and finish the handshake sequence. CURLOPT_TIMEOUT caps the entire transfer from first byte of the request to last byte of the response. Those are different clocks and setting only one of them produces the two classic failures.

A connect budget with no total budget lets a target that dribbles bytes forever hold a worker until the process dies. A total budget with no connect budget spends the whole allowance waiting on a dead node, so a 40 second ceiling becomes 40 seconds of nothing on every attempt against an endpoint that stopped answering.

My working pair is 6 and 40. Six seconds is generous for a server endpoint anywhere on the planet, and anything slower I abandon early. Forty seconds covers the slowest legitimate detail page on those domains, which sits at 11 seconds under load.

CURLOPT_CONNECTTIMEOUT_MS and CURLOPT_TIMEOUT_MS take the same budgets in milliseconds. They carry a trap. When the value falls below one second and libcurl was built against the standard resolver, it uses an alarm signal for the name lookup, and that signal interacts badly with anything else in a long lived PHP process. Setting CURLOPT_NOSIGNAL to 1 turns that off. I set it on every handle regardless of the timeout values, since the cost is nothing and the failure mode is a process that dies without explanation.

curl_setopt_array($ch, [
    CURLOPT_NOSIGNAL          => 1,
    CURLOPT_CONNECTTIMEOUT_MS => 6000,
    CURLOPT_TIMEOUT_MS        => 40000,
    CURLOPT_LOW_SPEED_LIMIT   => 512,
    CURLOPT_LOW_SPEED_TIME    => 15,
    CURLOPT_DNS_CACHE_TIMEOUT => 300,
]);

The low speed pair fixes a case neither timeout covers. CURLOPT_LOW_SPEED_LIMIT is a byte rate. CURLOPT_LOW_SPEED_TIME is a duration. Together they abort a transfer that stays under that rate for that long, which catches the stalled connection trickling one byte every few seconds to stay technically alive. Half a kilobyte per second over 15 seconds is my threshold and it has never fired on a healthy page.

The five timing fields in curl_getinfo come back as cumulative floats, and the useful numbers are the differences between neighbours: connect minus namelookup is the distance to the node, starttransfer minus appconnect is the target thinking.

curl_multi and a rolling window sized to the package limit

curl_multi is the parallel engine inside the extension, and the naive version of it is a trap. Adding 1400 handles at once means 1400 sockets attempted at once, a memory profile that climbs to whatever the response bodies weigh, and a concurrency figure far past anything your package permits.

The shape that works is a rolling window. Keep N handles in flight, and every time one finishes, harvest it and push the next URL into the gap.

function runWindow(array $urls, array $proxyOpts, int $window = 24): array
{
    $mh = curl_multi_init();
    curl_multi_setopt($mh, CURLMOPT_MAX_TOTAL_CONNECTIONS, $window);
    curl_multi_setopt($mh, CURLMOPT_MAX_HOST_CONNECTIONS, 8);

    $queue = array_values($urls);
    $live  = [];
    $out   = [];

    $push = function () use (&$queue, &$live, $mh, $proxyOpts) {
        if (!$queue) { return; }
        $url = array_shift($queue);
        $ch  = curl_init($url);
        curl_setopt_array($ch, $proxyOpts + [
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_NOSIGNAL       => 1,
            CURLOPT_CONNECTTIMEOUT => 6,
            CURLOPT_TIMEOUT        => 40,
            CURLOPT_ENCODING       => '',
        ]);
        curl_multi_add_handle($mh, $ch);
        $live[(int) $ch] = $url;
    };

    for ($i = 0; $i < $window; $i++) { $push(); }

    do {
        curl_multi_exec($mh, $running);
        if (curl_multi_select($mh, 1.0) === -1) { usleep(100); }

        while ($info = curl_multi_info_read($mh)) {
            $ch  = $info['handle'];
            $key = (int) $ch;
            $out[] = [
                'url'   => $live[$key],
                'errno' => $info['result'],
                'code'  => curl_getinfo($ch, CURLINFO_HTTP_CODE),
                'cconn' => curl_getinfo($ch, CURLINFO_HTTP_CONNECTCODE),
                'ip'    => curl_getinfo($ch, CURLINFO_PRIMARY_IP),
                'total' => curl_getinfo($ch, CURLINFO_TOTAL_TIME),
                'body'  => curl_multi_getcontent($ch),
            ];
            unset($live[$key]);
            curl_multi_remove_handle($mh, $ch);
            curl_close($ch);
            $push();
        }
    } while ($running || $live || $queue);

    curl_multi_close($mh);
    return $out;
}

Three details in that block carry the run. curl_multi_select blocks until something is ready, and the usleep(100) guard covers the case where it returns minus one immediately, which turns an otherwise pleasant loop into a spinning core. curl_multi_info_read is the only place a per handle error number appears, under the result key, since curl_errno on a handle already removed from the multi tells you nothing. And CURLINFO_HTTP_CONNECTCODE returns the status the endpoint gave to the CONNECT request, which separates a refusal at the node from a refusal at the target in one integer.

Now the window size. Two ceilings decide it. The first is the target: walk up from 8 and watch where 429 answers appear. The second is your package. My packages allow 1000 parallel connections, the corporate term goes up to 3000, they do not stack, and binding two addresses splits the allowance across them. A window above the ceiling produces queued work that reads like network slowness in every log you have.

The fourth row is where the curve flattens on my targets, and everything past it buys seconds while raising the refusal rate. Sustained work at those rows runs on an access period I renew every month, where the exit changes between calls on its own, so pacing stays in my code where I can see it.

Traffic volume never enters this arithmetic. Access on these terms carries no transfer ceiling, so concurrency is the only meter worth watching.

Guzzle's proxy option, keyed by the target scheme

Guzzle wraps all of the above and adds one idea that trips people on the first afternoon: the proxy array keys are the scheme of the target URI, and the values carry the scheme of the transport. Two scheme meanings in one short array.

use GuzzleHttp\Client;

$client = new Client([
    'base_uri'        => 'https://example.net',
    'proxy'           => [
        'http'  => 'http://A19f42:x7Qd21ka@203.0.113.24:8000',
        'https' => 'http://A19f42:x7Qd21ka@203.0.113.24:8000',
        'no'    => ['localhost', '127.0.0.1', '.internal', '169.254.169.254'],
    ],
    'connect_timeout' => 6.0,
    'timeout'         => 40.0,
    'http_errors'     => false,
    'headers'         => ['Accept-Encoding' => 'gzip, deflate'],
]);

$res = $client->get('/catalog/page/3');
echo $res->getStatusCode(), ' ', $res->getBody()->getSize(), "\n";

Set only the http key and every https:// request in the script leaves through your own address. Nothing raises, nothing warns, the target answers normally, and your log fills with successes from the address you were trying to keep out of the picture. I found this after 6000 requests, and the only reason I found it was an exit check that now runs on every worker.

A plain string applies to every scheme at once, and that is the form I use when there is nothing to split. The no key is the NO_PROXY equivalent, matching is suffix based on hostnames, and the metadata address in that list keeps cloud instance calls local.

SOCKS goes in through the same string. 'proxy' => 'socks5h://A19f42:x7Qd21ka@203.0.113.60:1080' reaches libcurl as CURLOPT_PROXY and the scheme sets the type there, which is why the client never needed a constant for it. When you do need a constant, the curl key takes it:

$res = $client->get('/api/items', [
    'curl' => [
        CURLOPT_PROXYTYPE     => CURLPROXY_SOCKS5_HOSTNAME,
        CURLOPT_PROXYUSERNAME => 'A19f42',
        CURLOPT_PROXYPASSWORD => 'x7Qd21ka',
        CURLOPT_NOSIGNAL      => 1,
    ],
]);

One naming difference deserves a sentence. Guzzle's timeout maps to CURLOPT_TIMEOUT_MS and covers the whole transfer. Guzzle's read_timeout applies only to the stream handler and does nothing under the cURL handler, which is the default on any build with the extension loaded. Setting read_timeout and expecting an idle guard gives you no guard.

One handler, one client, sockets that survive between calls

new Client() inside a loop is the most expensive line I have removed from other people's PHP. Each client builds its own handler stack, each stack builds its own CurlMultiHandler, and each handler owns its own connection cache. Throw the client away and you throw away every live socket with it, so the next request pays a TCP handshake, a TLS handshake and a CONNECT round trip before a single byte of your request leaves the machine.

Guzzle picks CurlMultiHandler automatically when curl_multi_exec exists. That handler holds one persistent multi handle for the life of the object, and libcurl keeps finished connections in its cache. Build the client once, keep it in a property, and the second call to the same host skips the entire setup sequence.

I measured it before believing it. 500 GET requests to one host, single process, warm DNS:

Four sockets on the middle row is the interesting figure. Three of them died mid batch on rotation events inside the pool and libcurl replaced them without a word, which is the behaviour you want and the reason the retry middleware below matters more than the connection cache itself.

The handler takes multi options directly, and that is where the concurrency ceiling lives under GuzzleHttp\Pool:

use GuzzleHttp\Client;
use GuzzleHttp\HandlerStack;
use GuzzleHttp\Handler\CurlMultiHandler;
use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Request;

$handler = new CurlMultiHandler([
    'options' => [
        CURLMOPT_MAX_TOTAL_CONNECTIONS => 24,
        CURLMOPT_MAX_HOST_CONNECTIONS  => 8,
    ],
]);

$stack  = HandlerStack::create($handler);
$client = new Client(['handler' => $stack, 'proxy' => $proxy, 'timeout' => 40.0]);

$requests = static function (array $urls) {
    foreach ($urls as $u) { yield new Request('GET', $u); }
};

$pool = new Pool($client, $requests($urls), [
    'concurrency' => 24,
    'fulfilled'   => function ($response, $i) use (&$ok) { $ok[$i] = $response; },
    'rejected'    => function ($reason, $i) use (&$bad) { $bad[$i] = $reason; },
]);

$pool->promise()->wait();

concurrency in the pool and CURLMOPT_MAX_TOTAL_CONNECTIONS on the handler are two different limits and both apply. The pool controls how many promises are in flight in PHP, the multi option controls how many sockets libcurl holds open. Setting the first high and the second low gives you a queue you cannot see. I keep them equal. Long jobs sit on the HTTP endpoints my overnight runs use, where the parallel connection figure is published with the package and goes straight into both numbers.

Middleware::retry, the decider and the delay

Guzzle has no retry policy out of the box. Middleware::retry gives you two closures and full control, which is better than a flag with an opinion baked in.

use GuzzleHttp\Middleware;
use GuzzleHttp\Exception\ConnectException;
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;

$decider = static function (
    int $retries,
    RequestInterface $request,
    ?ResponseInterface $response = null,
    ?\Throwable $e = null
): bool {
    if ($retries >= 4) { return false; }

    if ($e instanceof ConnectException) {
        $errno = $e->getHandlerContext()['errno'] ?? 0;
        return in_array($errno, [7, 28, 35, 52, 55, 56, 97], true);
    }

    if ($response) {
        return in_array($response->getStatusCode(), [408, 425, 429, 500, 502, 503, 504], true);
    }

    return false;
};

$delay = static function (int $retries, ?ResponseInterface $response = null): int {
    if ($response && $response->hasHeader('Retry-After')) {
        $after = (int) $response->getHeaderLine('Retry-After');
        if ($after > 0) { return min($after, 120) * 1000; }
    }
    return (int) (600 * (2 ** ($retries - 1)));   // 600, 1200, 2400, 4800 ms
};

$stack->push(Middleware::retry($decider, $delay), 'retry');

The decider signature carries both a response and a throwable, and exactly one of them is populated on any call. A ConnectException means nothing came back at all, and getHandlerContext() is where the raw cURL error number hides. That number is why the next section exists.

Three codes stay out of the repeat list on purpose. 403 comes from the target, and repeating a refusal adds another sample of the same behaviour to whatever is scoring you: one bad night turned a modest refusal rate into 9000 rejected requests in twelve minutes because 403 sat in that array. 404 means the page is absent. 407 comes from the endpoint and means your credential pair was declined, so every worker sees it on the next attempt too, and my handler raises immediately and stops the batch.

The delay closure returns milliseconds. A missing Retry-After falls back to a doubling series, and honouring the header when it arrives is the difference between a temporary throttle and an escalating one. The 120 second ceiling on that branch exists because I once read a header asking for 3600 seconds and watched a worker sit idle for an hour.

Worst case duration per URL is arithmetic worth doing once, since it explains slow runs better than any profiler. Five attempts at 46 seconds of budget each, plus 9 seconds of accumulated backoff, is 239 seconds on one URL holding a slot the entire time. With a window of 24 and a failure rate of 2 percent across 1400 pages, that tail adds nearly three minutes to a job that otherwise finishes in two.

cURL error numbers, read one at a time

curl_errno() returns an integer, curl_error() returns the text, and in a multi loop both arrive under the result key from curl_multi_info_read. Under Guzzle the same integer sits in the handler context of a ConnectException. Learning ten of these numbers by sight has saved me more time than any other habit in PHP networking.

Two rows need a sentence more. 5 and 7 look similar in a log and mean opposite things: the first says your machine never found the endpoint, the second says it found it and the door was shut. No amount of credential editing touches either one. The second is 97, which on a SOCKS endpoint covers a wide family, and the text alongside it names the member: Can't complete SOCKS5 connection with a method rejection means the endpoint wants no credentials at all, so bind the machine and drop the login from the string.

One thing is absent from that table on purpose. The 407 answer arrives as an HTTP status the endpoint returned on the CONNECT request, so curl_errno reads zero while CURLINFO_HTTP_CONNECTCODE reads 407. A checker that only looks at curl_errno reports a successful transfer on a batch that never reached a single target. Mine reads both integers on every request, and the pool behind it runs on server proxies on hardware we own, which keeps the CONNECT layer predictable enough that a 407 always means the credential and never the node.

A gateway in front, and the environment PHP hands to libcurl

Corporate networks put their own intermediary between your process and everything outside, and PHP inherits that arrangement through two paths.

libcurl reads proxy settings from the process environment when CURLOPT_PROXY is unset. The lowercase names are the ones it honours: http_proxy, https_proxy, all_proxy and no_proxy. The uppercase HTTP_PROXY is deliberately ignored, because in a CGI environment a client can send a Proxy: request header and see it appear in the process environment under exactly that name. That refusal is a security measure and it surprises people who set the uppercase form and watch nothing change.

Setting CURLOPT_PROXY to an empty string disables proxy use for that handle regardless of the environment. That one line is how I take a direct baseline on a machine whose shell profile exports a gateway without telling me.

$direct = curl_init('https://api.ipify.org');
curl_setopt_array($direct, [CURLOPT_PROXY => '', CURLOPT_RETURNTRANSFER => true, CURLOPT_TIMEOUT => 8]);
$mine = trim((string) curl_exec($direct));
curl_close($direct);

$through = trim((string) $client->get('https://api.ipify.org')->getBody());

if ($mine === $through) {
    throw new RuntimeException('traffic is leaving through the host address');
}

The exception is the point. A printed address you glance at proves nothing at three in the morning. A thrown exception stops the run.

Guzzle reads the environment on its own terms in configureDefaults. It picks up HTTP_PROXY only when the SAPI is cli, for the same reason libcurl ignores the uppercase form. It picks up HTTPS_PROXY and NO_PROXY in any SAPI. A script that behaves correctly from the command line and differently under FPM has this asymmetry underneath it more often than any code difference.

export https_proxy="http://A19f42:x7Qd21ka@203.0.113.24:8000"
export http_proxy="$https_proxy"
export no_proxy="localhost,127.0.0.1,.internal,169.254.169.254"

Under FPM those values come from the pool configuration through env[...] lines, and putenv() inside the script reaches libcurl too, which is a useful override in a worker that switches endpoints between jobs.

Chaining two hops comes up when a corporate gateway sits between you and the outside world. CURLOPT_PRE_PROXY sets a SOCKS hop that the connection passes through before reaching the HTTP endpoint in CURLOPT_PROXY, and the two options together give you a path with both. When the gateway speaks HTTP and demands NTLM, CURLOPT_PROXYAUTH with CURLAUTH_NTLM plus CURLOPT_HTTPPROXYTUNNEL covers it, and the tunnel flag matters because many such gateways refuse anything except CONNECT.

One habit closes the loop for me. Every run writes a journal line per request with the errno, the HTTP status, the CONNECT status, the exit address and the five timing floats. It costs one fputcsv call, and after two nights that file answers questions no single run can: whether failures cluster by endpoint or by target host, and how the refusal rate moves when the window grows by 8.

The neighbouring guides on this site carry the same work into other runtimes: the three JavaScript clients and their differing proxy agents are covered in axios, got and undici behind a proxy, the command line shape of every option above including the CONNECT trace lives in the curl flags reference, and the transport level version with its own connection pool sits in net/http and Colly in Go. When a dashboard line has to become a CURLOPT_PROXY string, a PROXYUSERNAME pair and a Guzzle array without a transcription slip at midnight, the connection string parser splits the four part form apart and prints each version ready to paste.