PROXIES BY TASK
Proxies for web scraping
Sites limit requests per IP. A pool of proxies spreads the load, so your scraper stops hitting captchas and bans.
A pool of datacenter IPv4 is enough for most tasks: fast, stable and inexpensive. Size the pool by the site limit: if it allows 300 requests per hour per IP and you need 3000, you need about 10 addresses.
IPv6 is the cheapest way to get many addresses, but only for sites that support IPv6. Check that before buying.
If a site aggressively blocks datacenters, switch to residential proxies: a new IP for every request and you pay only for traffic.
Which proxies to choose
Tips
- Add delays between requests and rotate User-Agents.
- Retry through another proxy on a 429 or a captcha.
- Log which IPs get refused and take them out of rotation.
FAQ
HTTP or SOCKS5 for a scraper?
HTTP is enough for web requests. Use SOCKS5 when your library or protocol requires it.