Headless Browser Performance Benchmarks for Web Scraping

The design decision made in each framework's founding year is what you are benchmarking today. That is not a metaphor. It is literally the code running in production.

Selenium arrived in 2004 and formalized its current architecture with the WebDriver protocol in 2011. Commands travel from your script to a driver process, then to the browser, then back again. That intermediary layer made sense before WebSocket was a viable alternative, and it reflected a philosophy rooted in interoperability: a standardized HTTP interface that could, in theory, talk to any browser through any conforming driver. The tradeoff was latency, baked into every single command, forever.

Puppeteer launched in 2017, built by members of the Chrome team, and it changed the model entirely. Instead of routing through a driver, Puppeteer opens a direct WebSocket connection to the browser via the Chrome DevTools Protocol. No intermediary, one persistent connection. CDP was already battle-tested by browser developer tools; Puppeteer simply exposed it for automation.

Playwright came out of Microsoft in 2020 and adopted the same CDP foundation for Chromium, then extended the approach to Firefox and WebKit through unified binaries the Playwright team maintains themselves. It also introduced built-in actionability checks: the framework waits for an element to be visible, stable, and interactable before acting on it, rather than leaving that burden to the developer. Depending on your workload, that addition is either a small tax you resent or the thing that stops your scraper from falling apart at two in the morning.

The architectural split, HTTP and WebDriver on one side versus WebSocket and CDP on the other, is the single biggest predictor of measured performance differences between these tools. Everything else is downstream of that.

What the HTTP-vs-WebSocket Gap Costs Per Command at Scale

Every Selenium selector resolution is a round-trip. Your script sends an HTTP request to the driver, the driver forwards it to the browser, the browser responds, the response travels back through the driver. On localhost, that adds something like 0.1 to 0.2 milliseconds per call. Sounds trivial. But run a scraper touching hundreds of elements per page across thousands of pages and those microsecond taxes compound into seconds of wasted wall time, and seconds compound into the difference between a job that finishes overnight and one that does not finish at all.

Playwright's persistent WebSocket connection eliminates that round-trip structure entirely. Every action is a single message over an already-open channel. Playwright also listens for navigation events via CDP rather than polling for them, and that difference is real: benchmarks put navigation operations at roughly 2.3 times faster than comparable Selenium flows.

The workloads where this gap hurts Selenium most are selector-heavy ones: table extraction, infinite scroll, dynamic forms with conditional fields. The more element interactions your script performs, the more that per-command overhead accumulates into something you actually feel.

Execution Speed Benchmarks Across Short Scripts and Real-World Scenarios

Checkly's 2024 benchmark found Puppeteer and Playwright running close to 20% faster than Selenium and WebDriverIO-style approaches in end-to-end scenarios. That headline number is fine, but the more instructive finding sits beneath it: Puppeteer is noticeably faster than Playwright on shorter scripts, and that advantage reverses in longer, more complex, real-world scenarios where Playwright eventually takes the lead.

This matters for architecture decisions more than people initially realize. If your scraper runs thousands of short, isolated page visits, Puppeteer's edge is real. For multi-step, stateful crawls with login flows, infinite scroll sequences, or multiple interactions per session, Playwright pulls ahead as the session lengthens.

In pages per minute, current estimates put Playwright somewhere around 35 to 55 under normal configuration, with Selenium trailing. Optimized headless Chrome setups, resource blocking enabled, parallel page loading, instance recycling, can reach well over 100 pages per minute on modest hardware.

Cold-start latency matters most when sessions are short-lived. Full browser-launch-plus-DOM-dump cycles clock in around 800 milliseconds to over a second for headless Chromium, somewhat faster for WebKit, and slowest for Firefox Headless. If your architecture spins up fresh browser instances for every request, those startup costs dominate your throughput ceiling regardless of which framework you picked.

Memory Footprint by Browser Engine and What It Sets as a Concurrency Ceiling

Chromium is the heaviest engine. Base memory before a single page loads runs between 80 and 120 megabytes, and each tab adds another 150 to 300 on top of that. Peak memory under real load can push past 500 megabytes. Firefox is lighter, WebKit lighter still, with a base footprint around half of Chromium's in favorable conditions.

Chromium also carries a cold-boot CPU cost the other engines do not impose at the same magnitude. Shader compilation and V8 initialization consume real resources before the browser renders a single character. On a machine running dozens of parallel workers, that startup overhead compounds in ways that do not show up in single-instance benchmarks.

WebKit, accessible only through Playwright, offers the smallest per-tab footprint of the three. On large-scale pipelines where per-instance cost multiplies across many workers, that difference translates directly into either higher concurrency on existing hardware or lower cloud spend for equivalent throughput. Firefox's footprint advantage over Chromium is meaningful in memory-constrained environments, even if WebKit takes the crown.

Running headful rather than headless costs more memory, more CPU, and slightly slower navigation due to UI rendering. For production scraping, headless is the correct default and not really a point worth debating.

Why Playwright's Browser Context Model Changes the Concurrency Math

Selenium's concurrency model is conceptually simple and practically expensive. One browser process per session, each carrying full memory overhead. Scaling to 20 concurrent sessions means 20 full browser instances, each bootstrapping independently.

Playwright's context model works differently: one browser process hosts multiple isolated contexts, each with its own cookies, storage state, and permissions, but all sharing the underlying process. Spawning a new browser context takes something in the neighborhood of 12 milliseconds. Spawning a new Selenium WebDriver session takes closer to 680. Across a 100-session scraping batch, that gap alone accounts for over a minute of session management overhead before you've loaded a single page.

The concurrency ceiling shifts accordingly. On an 8-core machine, Playwright can support a substantially higher number of concurrent workers via contexts than Selenium can with full browser instances before performance degrades. The gap is substantial.

One caveat that catches people off guard: each open page tab still spawns a Renderer Process. As context count grows, CPU scheduling becomes the next bottleneck after memory. The right architecture for Playwright-based concurrency is context pooling, not instance pooling. Spawning a new full browser instance per incoming request creates massive redundancy in GPU processes and Network services that will eventually eat your throughput gains.

How Parallelism Strategy Translates to Throughput at Scale

Parallel browsers compress sequential scraping work in a way that no amount of per-command optimization can match. Pages that would take hours in a single-threaded loop can finish in minutes when fetched concurrently across a pool of workers.

One documented migration case: a team moving from Selenium to Playwright on an 8-machine CI setup saw a 60% reduction in run times. The gains came primarily from parallelism improvements via the context model, not from raw per-page speed differences. That ratio shows up consistently. Session management overhead is often larger than the per-command latency gap, and most people underestimate it until they've profiled a real workload.

Vendor-reported figures from managed platforms suggest throughputs in the range of tens of thousands of pages per hour with sub-2-second response times. Treat those numbers as directional. They are not guarantees replicable on self-managed infrastructure with a general-purpose machine.

There is, however, a ceiling that raw concurrency cannot overcome: bot detection. Hitting a target with 50 simultaneous headless browsers does not just stress your own infrastructure; it signals non-human behavior to any competent anti-bot system. Throttle concurrency to the target site's tolerance, not just to available hardware. The right parallelism level is a function of the target, not the machine.

How Anti-Bot Systems Detect Headless Browsers and Which Framework Artifacts Are Hardest to Hide

The obvious detection signals are well known: the navigator.webdriver flag, implausible screen resolutions, missing WebGL signatures, GPU fingerprints characteristic of data center rendering environments like SwiftShader or llvmpipe. Most stealth configurations address these. They are not the hard problem.

The hard problem is CDP artifacts. CDP-specific event signatures appear in browser performance timelines. Layout and paint events fire with timing offsets characteristic of automated execution, and these signatures survive surface-level JavaScript patches because they originate at the transport layer, not in any JavaScript-accessible API. You cannot patch your way out of a timing artifact that lives below the JS runtime. I've watched teams spend weeks on stealth configurations that addressed everything except this and wondered why hardened targets kept catching them.

Running Playwright with default settings is detected immediately by hardened platforms. Regular stealth configuration updates, custom fingerprints, and residential proxy infrastructure are baseline requirements for operating against serious anti-bot deployments, not advanced optimizations.

Proxyway's 2025 benchmark across 15 heavily protected sites illustrates the variance clearly. The leading provider achieved a 93.14% success rate at 2 requests per second; the hardest target, Shein, averaged a 21.88% success rate across all providers. That spread tells you something important: target-site difficulty dwarfs framework choice as a success-rate variable at the upper end of protection tiers.

One counterintuitive implication worth sitting with: the resource optimizations that reduce memory and improve throughput can simultaneously make your fingerprint more anomalous. A browser with images, fonts, and media blocked looks nothing like a real user session. The more aggressively you strip browser features, the more detectable the resulting fingerprint becomes on hardened targets.

Resource Optimization Techniques and the Tradeoffs They Introduce

The primary levers are well established. Blocking images, fonts, and media resources can reduce per-tab memory by 40 to 60%. Using waitUntil: 'domcontentloaded' instead of networkidle cuts wait time on pages where you do not need all network activity to resolve. Closing tabs immediately after use rather than holding them open prevents memory from accumulating across a long-running job. Always calling browser.close() in a finally block is mandatory; orphaned browser processes will eventually exhaust your system resources, and this is the kind of thing that bites you at 3 AM when a job has been running for six hours.

Batch operations, processing multiple URLs within the same browser session rather than launching fresh instances, amortize cold-start costs across many page visits. On workloads dominated by cold starts, this single change can halve total execution time. Reducing custom viewport size means the browser renders less content per tab, which chips away at per-tab memory without much visible downside for most scraping tasks.

Combined, these optimizations can reduce headless Chromium's resource usage substantially while maintaining full functionality for most workloads. But every reduction in browser fidelity is also a reduction in how closely the session resembles a real user. Aggressive optimization and effective stealth on hardened targets are, to a real degree, in opposition with each other. Figure out which constraint is actually binding on your workload before you start tuning, because optimizing for the wrong one makes both problems worse.

Matching Framework and Configuration to Workload Type

Short, isolated page visits at high volume: Puppeteer's speed advantage on short scripts is genuine, and Chromium-only is a reasonable constraint if cross-browser parity isn't a requirement. Pair it with context pooling to contain memory costs.

Multi-step, stateful crawls with login flows, infinite scroll, or form interactions: Playwright's performance advantage grows with session complexity, and its built-in actionability checks prevent an entire class of timing bugs that otherwise require custom retry logic. The slightly higher overhead versus Puppeteer is a reasonable price for the stability.

Memory-constrained infrastructure or large instance counts: WebKit via Playwright offers the smallest per-tab footprint of any supported engine. Firefox is the alternative where WebKit compatibility is a concern for a specific target.

Legacy codebases or teams already invested in Selenium: Selenium 5.0's architectural improvements narrow the gap, and migration carries real costs. For net-new work, the protocol overhead is a concrete, measurable disadvantage that compounds at scale.

Heavily protected targets: framework choice becomes a secondary consideration. Stealth configuration, proxy quality, and fingerprint management determine outcomes at this tier. The infrastructure around the browser matters more than the browser itself, and Proxyway's 2025 data, that spread from 93% to 22% success rates by provider, makes that point as clearly as any benchmark I've seen.