Calculators and guides for proxy work
Four tools we kept rebuilding in spreadsheets, turned into pages that answer in a second. Then the guides that explain the numbers behind them.
Pool and thread calculator
Work out how many addresses and concurrent threads a collection job needs from its volume, time window and the safe rate per add
converterConnection string converter
Paste a proxy line in any common format and get ready snippets for curl, Python, Node, Playwright, Scrapy, environment variables
referenceResponse code reference
Filterable reference of the codes and connection errors you meet through a proxy, with the layer that issued each one and the ac
plannerRate and pause planner
Turn a daily page target into a request rate, a pool size and a pause range with enough spread that intervals stop looking machi
Guides
Proxifier SOCKS5 Setup
Windows has no single switch that pushes every program through a proxy. Chrome picks up the system settings, curl reads
guideSwitchyOmega proxy setup
SwitchyOmega sits between Chrome and the browser proxy configuration, and it decides one thing that matters: which reque
guidePlaywright proxy setup
Playwright hands you three places to attach a proxy, and the place you pick decides what the rest of the run is able to
guidePuppeteer proxy authentication
I keep a small fleet of Puppeteer workers that pull catalog pages and search results into my own reports. The first vers
guideAdsPower proxy setup
I keep 60 to 80 AdsPower profiles alive at any given time. The browser handles the fingerprint side of that work. The ad
guideSelenium proxy authentication
I keep a set of Selenium jobs that sign into supplier portals and export order tables nobody publishes through an API. T
guideScrapy proxy middleware
Scrapy ships with proxy support already wired in. Most people meet it once, set an environment variable, watch a crawl w
guidecurl through a proxy
Almost every proxy question I have chased this year was answered by one curl command typed into a terminal. The command
guidePython requests proxy session retry
My first proxied script in Python was four lines long. One requests.get, one proxies dict, one print. It worked, I felt
guideNode.js HTTP request through a proxy
Three HTTP clients cover most of the Node code I maintain. axios sits in the older services, got runs the collection scr
guideGolang HTTP proxy transport
Go hands you a proxied request in three lines and then leaves you alone with everything that follows. My first version w
guidePHP cURL proxy
My collection code in PHP started as a single curl_setopt call and one hard coded endpoint string. It fetched a page, pr
guideHow to tell what blocked my request
A status code gives you the verdict. The header block above it names the party that issued the verdict, and those are tw
guidemitmproxy inspect requests
The request that leaves your socket and the request your code describes are two different objects. Libraries add headers
guideProxy checker script
Every collection job I run starts with a text file. A few hundred lines of IP:PORT, sometimes IP:PORT:LOGIN:PASS, pulled
guideGitHub Actions proxy setup
One of my nightly workflows pulls about 4 000 catalogue pages and drops them into a bucket. It ran green for eleven nigh
guideScrape product pages at scale
Last quarter I took a catalogue with a shade over 200 000 items. The brief read like every other one: title, price band,
guideScrape job boards
I keep a daily picture of one national job aggregator for a labour analytics group. The open set sits around 412 000 pos
guideScrape business directories
A regional directory reached me with a plain brief. Every organisation in the city as one row, with phones, hours and an
guidePrice monitoring pipeline
I keep a price monitor running for an appliance and power tool retailer. Forty thousand of our own positions, nine compe
Why these four
Every collection job runs into the same four questions. How many addresses does this volume need. What does the connection line look like in the tool I actually use. Who returned this code. How fast can I go without the platform noticing. Answers to all four used to live in scattered notes, so they were rebuilt here as pages that take input and return numbers.
The arithmetic assumes a private pool where the allowance of an address belongs to you: private server addresses with no outside load, and for collection work specifically a pool sized for data collection. On a shared pool the calculations stop forecasting anything, because somebody else's rate is added to yours.