BT

Everything at Google Runs in Containers

by Carlos Sanchez on Jun 06, 2014 |

Google is using containers to run everything in their clusters, starting over 2 billion of them per week.

Joe Beda, senior staff software engineer at Google Cloud Platform, gave a presentation at Gluecon about how the company is using Linux container technologies, stating that everything at Google runs in a container. The result is that Google starts over 2 billion containers per week, over 3000 started per second, not counting long-running containers.

Google is using container technologies since 2004, releasing cgroups in 2006 and just last year created the project Let Me Contain That For You (lmctfy), the open source version of Google’s container stack, which provides Linux application containers replacing LXC. These containers allow for the isolation of resources used by multiple applications running on a single machine and gives the applications the impression of running exclusively on a machine. The applications may be container-aware and thus be able to create and manage their own subcontainers.

Rohit Jnagal, software engineer at Google, commented on lmctfy:

We have lmctfy managing all of Google's resource isolation needs since 2007. So far, it was mangled into other pieces of Google infrastructure. During a redesign, we were able to separate this layer out cleanly and thought it would be fun to put it out and give back.

He also describes the main differencies between lmctfy and LXC:

  1. Resource management API: LXC API is built for namespace support and exports cgroup support almost transparently. With lmctfy, Google tries to provide an intent-based resource configuration without users having to understand the details of cgroups, whose API may be unstable and hard to deal with.

  2. Priority: lmctfy is built to provide support for resource sharing and for overcommitting machines with batch workloads that can run when the machine is relatively idle. All applications specify a priority and latency requirements and lmctfy manages all cgroup details to honor the requirements for each task.

  3. Programmatic interface: lmctfy is the lowest block of app management for Google's cloud. It is built to work with other tools and programs, and much better specified and stable for building more complicated toolchains above it.

Google is also integrating containers into the Google Cloud Platform. The node container manager, open sourced by Google, is a small Python agent designed to manage a group of Docker containers according to a YAML manifest. An open preview version to manage containers running on Google Compute Engine is already available, supporting container manifests and the creation of containers at startup. The stack is based on Debian 7 and includes the Docker runtime and the node container manager.

Docker will also be able to take advantage of lmctfy since there is already an early integration for the Docker lmctfy driver, thanks to the new ability to run containers in environments other than LXC included since Docker version 0.9.

Rate this Article

Relevance
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

2 billion per week? by Cameron Purdy

Apparently they're limited in the number of containers because they chose to use a signed 32-bit integer ;-)

(j/k)

2 billion per week? by Cameron Purdy

Apparently they're limited in the number of containers because they chose to use a signed 32-bit integer ;-)

(j/k)

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

2 Discuss
General Feedback
Bugs
Advertising
Editorial
Marketing
InfoQ.com and all content copyright © 2006-2016 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT

We notice you’re using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.