Why your residential proxy bandwidth drains so fast (and how to cut it)

Residential proxy plans look cheap until the first invoice. You pay for every byte that travels through the proxy, in both directions. That includes images, video, fonts, tracking scripts, retries and redirects, not just the data you keep. Most overspending comes from three habits: rendering pages you don't need to render, re-downloading pages that haven't changed, and sending everything through residential IPs, including requests that don't need them.

Step 1: measure before you optimize

The formula is simple:

monthly GB = pages per run × runs per month × average MB per page ÷ 1024

The unknown is "average MB per page", and it varies enormously between "plain HTML" and "full browser load". Measure it for your target:

import requests

r = requests.get("https://example.com/product/123",
                 headers={"Accept-Encoding": "gzip, br"}, timeout=20)
print("decompressed HTML:", round(len(r.content) / 1024), "KB")
print("on the wire (if sent):", r.headers.get("Content-Length"), "bytes")

For a browser-based scraper, open the page with DevTools → Network and read the "transferred" total at the bottom. That figure, not the HTML size, is what a headless browser pulls through your proxy.

Step 2: see where the money goes

An illustrative example. Plug in your own measurements:

The same data, collected four different ways, differs by roughly 50× in bandwidth. That's why the provider you choose matters less than how you scrape.

Step 3: the cuts, from biggest to smallest

Don't render when you don't have to

Many "JavaScript-only" sites embed their data in the initial HTML, inside a <script type="application/ld+json"> block or a __NEXT_DATA__ / state object. Others load it from a JSON endpoint you can see in DevTools → Network → Fetch/XHR. Where the site's terms allow it, requesting that endpoint directly is often 10–20× lighter than rendering the page.

Block heavy resources in headless browsers

# Playwright (Python, async)
BLOCK = {"image", "media", "font", "stylesheet"}

async def lean(route):
    if route.request.resource_type in BLOCK:
        await route.abort()
    else:
        await route.continue_()

await page.route("**/*", lean)
await page.goto(url, wait_until="domcontentloaded")

Also block known third-party analytics and ad domains. They add weight and requests without adding data. Test before you deploy: a few sites check whether CSS loaded, and blocking stylesheets breaks them.

Ask for compressed responses

Send Accept-Encoding: gzip, br. Most HTTP clients do this by default, but some custom setups and older libraries don't, and uncompressed HTML can be 5–10× larger on the wire.

Don't download what hasn't changed

• Use conditional requests (If-None-Match with the stored ETag, or If-Modified-Since). A 304 Not Modified costs almost nothing.
• Check the site's XML sitemap <lastmod> dates and re-scrape only URLs that changed.
• Scrape by volatility: prices on best-sellers hourly, long-tail catalog pages daily or weekly.

Route by protection level

Sitemaps, images on a CDN, public APIs and unprotected sections often work fine on datacenter proxies costing a fraction per GB, or nothing beyond the IP price. Keep residential bandwidth for the requests that actually get blocked without it. See the proxy types guide.

Stop retry storms

A blocked page retried ten times costs ten times as much. Cap attempts, back off exponentially, and bench IPs that fail, as in the Python rotation pattern. Log the failure rate per target: if it climbs, fix the scraper instead of paying for more retries.

Deduplicate URLs

Tracking parameters (?ref=, utm_, sort orders) can make one product look like five URLs. Normalize URLs before queuing them.

Step 4: pick a pricing model that fits

• Pay-as-you-go traffic that doesn't expire is best for irregular jobs. You don't lose unused GB at month end.
• Monthly plans get cheaper per GB at higher tiers, so only commit once you know your real usage.
• Scraping APIs (pay per successful request) can beat per-GB pricing on heavy, well-protected pages, because failed attempts and page weight become the provider's problem.