Proxy Toolbox
Proxy Toolbox / Tools / Pool and thread calculator
calculator

Pool and thread calculator

Give it the volume, the window and the safe rate you measured. It returns the number of addresses, the thread count and the runtime, and tells you when the job runs past the thread cap of a package.

Total URLs in the job
How long the run may take
Measure once per platform
Median, not best case
Failed requests you repeat
Packages do not stack
0requests per minute needed
0addresses at the safe rate
0concurrent threads
0hours at your cap
Thread cap used0%
Requests with retries0

How the numbers are produced

Nothing here is a guess. Total requests come from the page count plus the share you expect to repeat. Divide that by the window in minutes and you have the rate the job demands. Divide the rate by the safe rate of a single address and you have the pool size. Threads follow from the rate and the median response time, because a request occupies a slot for exactly as long as it takes to answer.

Measuring the safe rate once

Run 200 requests from one address against the section you plan to collect, raising the pace in steps until the first refusal, then take about 60 percent of that figure. The measurement lives in a table afterwards and gets reused for every job on that platform. Sections differ: a product page tolerates far more than a live price endpoint.

Where the thread cap comes from

The cap belongs to the package, not to the pool. A standard package carries 1000 threads and a corporate one up to 3000. Two packages do not add up, and binding two addresses to a package splits the allowance between them. When the calculator says the job wants more threads than the cap allows, the honest options are a wider window or a second package. Details of the packages sit on the page for packages with no traffic counter.

Why the reserve matters

The retry share is the field people set to zero and regret. Refusals arrive in bursts, and a run sized to the exact minimum falls behind on the first heavy day. Twelve percent is a reasonable starting point for a marketplace, higher for search pages. Addresses for this kind of load come from a private pool built for data collection, where the allowance of an address belongs to you alone and the arithmetic stays predictable.