BT

Your opinion matters! Please fill in the InfoQ Survey!

How NGINX Achieves Performance and Scalability

| by Sergio De Simone Follow 5 Followers on Jun 12, 2015. Estimated reading time: 2 minutes |

A note to our readers: As per your request we have developed a set of features that allow you to reduce the noise, while not losing sight of anything that is important. Get email and web notifications by choosing the topics you are interested in.

Owen Garrett, heads of products at Nginx, Inc., has described on Nginx’s blog which design decisions allow NGINX to provide top-in-class performance and scalability.

The overall architecture of NGINX is characterized by a collection of processes that work together:

  • Master process: responsible to perform privileged operations such as reading configuration files, binding sockets, and creating/signalling child processes.
  • Worker processes: responsible for accepting and serving connections, reading and writing to disk, and communitating with upstream servers. These are the only processes that are busy when NGINX is active.
  • Cache loader: responsible for loading the disk cache into memory. This process runs at startup and then exit.
  • Cache manager: responsible for pruning entries from the disk caches to stay within their boundaries. It is run periodically.

The key to NGINX performance and scalability lies with two fundamental design choices:

  • The number of worker processes is constrained to minimize context switching. The default and recommended configuration is using one worker process per CPU core to make an efficient use of hardware resources.
  • Worker processes are single-threaded and handle multiple connections in a non-blocking fashion.

Each worker processes in NGINX handles multiple connections through a state machine that is implemented in a non-blocking fashion:

  • A worker process has got a number of sockets to handle, they are either listen or connection sockets.
  • When a new request comes in on a listen socket, a new connection socket is opened to handle communication with the initiating client.
  • When an event comes in on a connection socket, the worker responds promptly to it and moves to handle any other events that has come in on any sockets.

NGINX design choices, Garrett says, make it depart radically from other web servers, which usually opt for a model in which each connection is assigned a separate thread. This makes it very easy to handle multiple connections, since each connection can be thought of as a linear sequence of steps, but has a cost in terms of context switching. Indeed, worker threads spend most of their time in blocked state, waiting for the client or some other upstream server. The context switching cost becomes non-trivial when the number of simultaneous connections/threads that want to, e.g., execute I/O operations grows beyond a threshold, or when memory gets exhausted.

NGINX design, on the other hand, makes the workers never block on network traffic, unless there is no work to be done. Additionally, every new connection consumes very little resources, just a file descriptor and a small amount of memory in the worker process.

Overall, this allows NGINX to be able to handle up to hundreds of thousands of concurrent HTTP connection per worker process with appropriate system tuning.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT