Nginx at Scale

Nginx at Scale: Capability, Architecture, and Who Actually Uses It

Nginx Fundamentals and Nginx for Real-Time Chat, Video Streaming, and SSE are both about configuring a single nginx instance correctly. This essay is about a different question: what does it actually mean to run nginx "at scale," what nginx is genuinely good at once you get there, where that capability runs out, and who really uses it in production.

Table of Contents

1. What "At Scale" Actually Means
2. The Architecture That Makes It Possible
3. What Nginx Is Actually Capable Of, at Scale
4. How Nginx Itself Scales Horizontally
5. Who Actually Uses It
6. Where the Capability Runs Out
7. Synthesis

What "At Scale" Actually Means

"Scale" gets used as if it were one number, and that's the first thing worth unlearning. A server under real load is being stretched along several independent axes at once, and nginx behaves differently on each one:

• Request throughput — how many requests per second can be served. This is mostly a function of the event-driven architecture covered below, and it's the axis nginx is most famous for.
• Concurrent connections — how many sockets can stay open simultaneously, as distinct from how fast they're each being served. A chat or SSE fleet (from the Real-Time guide) can be under enormous connection pressure while under almost no throughput pressure at all — tens of thousands of nearly-idle sockets.
• Configuration surface area — how many distinct services, hostnames, and routing rules a single fleet has to represent. A company with three services behind nginx and a company with three hundred are running the same software, but living in completely different operational worlds.
• Geographic distribution — whether "scale" means one very large server, a cluster of servers in one datacenter, or hundreds of edge nodes spread across continents, each needing to agree on configuration and certificates.
• Rate of change — how often the set of backends, routes, or certificates actually changes. A monolith with three backend pools that haven't changed in a year scales differently than a Kubernetes cluster where pods are rescheduled every few minutes.

A single nginx instance, tuned the way the Fundamentals guide describes, can genuinely saturate the first two axes on commodity hardware — that part of the reputation is earned, not marketing. The last three are where "nginx at scale" stops being about nginx.conf and starts being about everything built around nginx.

The Architecture That Makes It Possible

The event-driven, worker-per-core model from Nginx Fundamentals is worth re-stating here specifically because it's the actual root cause of nginx's scaling story, not an incidental detail. A thread- or process-per-connection server (the model nginx was explicitly built to replace) pays a real, fixed memory and scheduling cost for every connection it holds open, whether or not that connection is doing anything. Ten thousand idle keep-alive connections cost roughly the same as ten thousand active ones. Nginx's workers instead register interest in a socket with the kernel and do nothing until the kernel says there's actual work — an idle connection costs a small amount of memory and essentially zero CPU.

This is precisely why the "C10K problem" (serving ten thousand concurrent connections on one machine) was treated as a serious engineering challenge in the early 2000s, and why nginx — along with a handful of contemporaries built the same way — is the reason it stopped being one. It's also why, when people say nginx "scales," they very often mean something more specific than they realize: not that nginx makes more CPU cycles appear, but that it wastes almost none of them on connections that aren't currently asking for anything.

What Nginx Is Actually Capable Of, at Scale

Four roles, each of which shows up in real large-scale deployments, and each of which stresses a different part of nginx:

Reverse proxy and API gateway at the edge of a service-oriented architecture. The microservices routing pattern from Nginx Fundamentals — one hostname, many location blocks, each forwarding to a different backend — is exactly how organizations with dozens or hundreds of internal services present them as one coherent API surface. At scale, the interesting part isn't the proxying itself (that's cheap), it's that the routing table — which paths go where — becomes a piece of infrastructure in its own right, usually generated by something else rather than hand-edited (more on this below).

TLS termination for an entire fleet. Terminating TLS at nginx instead of in every application process is a well-established pattern, and it genuinely offloads real CPU work (asymmetric cryptography for the handshake) away from application servers. At scale, this introduces a problem that doesn't exist on a single box: ssl_session_cache shared:SSL:10m is shared across the workers of one nginx instance, not across a fleet of them. A client whose TLS session was cached on node A but gets load-balanced to node B on reconnect can't resume cheaply there — it pays for a full handshake again. Fleets that care about this either terminate TLS one layer further out (a cloud load balancer or CDN in front of the nginx fleet, which is increasingly the default architecture), or distribute a shared TLS ticket key across all nginx nodes so any of them can resume a session encrypted by any other — a real piece of operational machinery that a single-box deployment never has to think about.

A caching tier in front of origin servers. proxy_cache (from the caching section of Nginx Fundamentals) turns nginx into something close to a purpose-built, self-hosted CDN node: a layer that absorbs repeat requests and shields the actual application from load it doesn't need to see. At scale, this is frequently deployed in layers — a tier of caching nginx nodes close to users, in front of a smaller tier doing the real proxying and TLS work, in front of the application itself — rather than one undifferentiated fleet doing everything.

Static and media asset serving. sendfile, aio/directio, and range-request support (covered in the Performance Tuning and Video Streaming sections) are what let a modest number of nginx nodes serve an amount of bandwidth that has nothing to do with request count and everything to do with disk and network throughput — this is the role nginx plays inside Netflix's Open Connect appliances, discussed below, where the entire job is pushing as many video bytes as the network card allows.

One role that deserves its own callout because it's arguably the single most common "nginx at scale" deployment today, and it's easy to miss if you're thinking in terms of hand-written config files: nginx as a Kubernetes Ingress controller. ingress-nginx takes the exact reverse-proxy and TLS-termination capability described above and puts a control loop in front of it: Kubernetes Ingress resources describe the desired routing in a declarative API, and the controller continuously regenerates and reloads nginx's configuration to match. Every one of the hundreds of routing rules a large cluster might have is still, underneath, an ordinary location block — the capability is identical to what's in Nginx Fundamentals. What's different is that no human is editing nginx.conf by hand anymore; the config has become a compilation target for a higher-level system. This is the direct answer to the configuration-surface-area axis above: at real scale, the question stops being "how do I write this nginx config" and becomes "what generates this nginx config, and how fast can it regenerate it correctly."

How Nginx Itself Scales Horizontally

A single, well-tuned nginx instance on modern hardware can hold hundreds of thousands of connections and comfortably serve tens of thousands of requests per second for typical workloads — genuinely enormous numbers for one machine. But "at scale" deployments almost never bet everything on one instance being big enough, for reasons that have nothing to do with raw capacity: a single machine is a single point of failure, a single point of maintenance-window downtime, and a single geographic location.

The actual pattern is close to embarrassingly simple: run many identical, stateless nginx instances, and put something else in front of them to spread traffic across the fleet — DNS round robin, a cloud provider's own load balancer (which handles the TLS-session-resumption and health-check concerns at a layer nginx itself doesn't span), or, as above, a Kubernetes Deployment of ingress-nginx pods behind a Service, scaled up and down by the same autoscaler that manages everything else in the cluster. Nginx itself contributes very little special-casing to this pattern — it's horizontally scaled the same way any stateless HTTP service is — which is itself worth noticing: nginx's scaling story at the fleet level is almost entirely "be easy to run many identical copies of," not some distinctive clustering feature of nginx itself. (Nginx Plus adds a few fleet-aware features — a shared key-value store and dynamic upstream reconfiguration via an API — but the open-source core deliberately doesn't try to be its own orchestration layer; it expects one to exist above it.)

Who Actually Uses It

The honest framing is less "which famous logos run nginx" and more "which architectural role is nginx filling, and does your organization have that role somewhere." That said, the specific, public examples are worth knowing because they illustrate the range:

• Netflix runs nginx inside its Open Connect appliances — the CDN hardware Netflix deploys directly into ISP networks around the world specifically to serve video close to viewers. This is publicly documented in Netflix's own engineering writing, and it's the "narrow and deep," bandwidth-bound use case from the Real-Time Streaming guide taken to its logical extreme: appliances whose entire job is pushing as many video bytes as the hardware allows.
• Dropbox, Airbnb, and Automattic (WordPress.com) have all published engineering material over the years describing nginx as a core part of their edge and reverse-proxy layer — exactly the API-gateway-at-the-edge-of-many-services role described above, at a scale of many millions of users.
• Cloudflare is the most interesting case precisely because it isn't a simple "they use nginx" story. Cloudflare's edge ran on heavily patched nginx for a large part of its history — a genuine, large-scale validation of the architecture — and then, publicly and in detail on their own engineering blog, they built and migrated to Pingora, a proxy written from scratch in Rust, specifically because their scale had reached a point where nginx's process/configuration-reload model and per-connection memory overhead had become the limiting factor rather than an advantage. That's a company that scaled past nginx, which is at least as informative as a company that scaled on it — and it's a useful preview of the honest limits covered next.
• Beyond named examples, public web-server market-share surveys (W3Techs and Netcraft both publish ongoing data of this kind) have for years consistently shown nginx as one of the one or two most-used web servers among the highest-traffic sites on the internet specifically — a different and arguably more meaningful statistic than its share of all websites, since it reflects what operators choose once traffic actually gets demanding. Treat the exact current figures as something to look up fresh rather than take from memory here, since survey numbers shift and this is exactly the kind of claim worth verifying against the live source before repeating it in anything that matters.

The more useful takeaway than any individual logo: almost any organization running a non-trivial number of backend services behind one public surface — whether that's a hand-rolled fleet of reverse proxies or a Kubernetes cluster using ingress-nginx by default — has nginx somewhere in its request path, frequently without anyone needing to make a deliberate, conscious choice to be "an nginx shop."

Where the Capability Runs Out

A genuine essay about capability has to name the edges, not just the strengths — and nginx has real ones, which is exactly what the Cloudflare/Pingora example above previews.

CPU-bound work still blocks a worker. The event loop that makes idle connections nearly free makes the opposite mistake expensive: since each worker is single-threaded, a slow, CPU-heavy operation on one connection — a large response body rewrite, an expensive regular expression, a complex Lua script via the OpenResty/ngx_http_lua_module ecosystem — blocks every other connection that same worker is holding, for as long as it takes. A thread-per-connection server has the opposite trade-off: a slow request only ties up its own thread. Nginx's performance story is specifically an I/O-bound story; it doesn't automatically parallelize CPU-bound per-request logic the same way.

The reload model assumes configuration changes slowly. nginx -s reload — spawn fresh workers with the new config, drain the old ones — is a genuinely elegant way to apply config changes without dropping connections, and it's fine for the rate of change most organizations actually have (deploys a few times a day, certificate renewals every couple of months). It was never designed for per-request routing decisions that change thousands of times a second, which is precisely the problem a service mesh's sidecar proxy has to solve: service discovery where backend pods come and go every few minutes, with routing rules that need to take effect in milliseconds, not after a reload cycle. This is a large part of why Envoy — built from the start around a dynamic, streaming configuration API (xDS) rather than a static file reloaded periodically — displaced nginx in a lot of service-mesh and sidecar-proxy deployments, and it's the same underlying limitation Cloudflare cited for building Pingora instead of continuing to stretch nginx further.

Dynamic upstream membership isn't native to the open-source core. Stock nginx's upstream {} block is a static list, re-read on reload; picking up a newly added backend without a reload requires either a templating sidecar that rewrites and reloads the config on every change (confd, consul-template, or exactly what ingress-nginx does internally), nginx's own resolver directive combined with variables to force periodic DNS re-resolution, or Nginx Plus's commercial dynamic-upstream API. None of these are wrong, but all of them are evidence that "automatically discovering and routing to the right backend as the fleet changes" is something built on top of nginx, not a capability nginx ships with out of the box.

Synthesis

Nginx's reputation at scale is earned, but it's earned for a narrower reason than "it can do everything": one architecture — event-driven, single-threaded-per-worker, non-blocking I/O — executed extremely well, for exactly the workload that architecture suits (serving and proxying HTTP traffic with high connection counts and mostly I/O-bound work). Every genuinely large deployment covered here is really a story about pairing that one strength with something else built around it: a cloud load balancer or CDN in front for the problems that span more than one box, a control-plane like ingress-nginx's controller or a config-templating tool for the problem of configuration changing faster than a human can hand-edit it, and — at the point where request-level routing needs to change thousands of times a second or per-request logic gets CPU-heavy enough to matter — sometimes a different piece of software entirely, which is exactly the decision Cloudflare and plenty of service-mesh architectures have made in public. Knowing which of those problems is actually yours, rather than assuming nginx alone solves all of them, is most of what "running nginx at scale" turns out to mean in practice.