Unix Orchestration Roundup: Tools for Programmatic Systems Administration

Posted by Chris Brown on May 02, 2011

Sections

Enterprise Architecture,

Operations & Infrastructure,

Development

Topics

Puppet ,

Devops ,

IT Service Management ,

Orchestration ,

Infrastructure ,

Operations ,

Cloud Computing

Share |

RELATED Research

While system administrators have always written code to automate tasks and many developers run their own systems, the melding of development and operations into devops has really taken hold as sophisticated configuration management and orchestration tools have become available.

For purposes of this roundup, we focus on applications that perform orchestration -- ranging from simply running a one-liner on a bunch of machines to coordinating rolling restarts as a new version of an application becomes available.

The choice of orchestration platform is largely based on the platform ecology of the systems to be orchestrated. In the Linux space, several lightweight, non-commercial options are available, including:

ControlTier can do orchestration and lots more, but development has slowed; a key contributor recently forked it into RunDeck as “a spin off or a promotion of some of ControlTier's most popular features”. In addition to ad hoc orchestration -- simply “run this command on these servers” -- RunDeck can be used for complex workflows and automation, such as “at 10pm, start bringing down servers until there is only one left up, then deploy the patch, then restart the downed servers”. RunDeck can be integrated with Puppet for configuration management. Both products are Apache-licensed.

Fabric is popular in the Python community and is BSD-licensed. Designed for “streamlining the use of SSH for application deployment or systems administration tasks”, it’s a natural fit when your goal is to execute a simple script on lots of servers. At the same time, since it’s a programming library, it can be embedded into more complex programs.

Func is closely tied to Red Hat and its derivatives, including CentOS. The project is GPLv2-licensed recently released its 0.28 version. Func lacks some of the dynamic nature of other orchestration tools; in order to run a command on a related set of servers, a central authority must define server groups. This centralized inventory is easy to capture in SCM but perhaps redundant with inventories maintained in a configuration management system.

Chef, from OpsCode, comes with Knife. As its name implies, it’s a tool used heavily by Chef, but it can run independently of Chef (for example, those who use Puppet for configuration management can use Knife); like all of the other tools, it can be used to run arbitrary commands over SSH. It is Creative Commons-licensed.

Puppet Labs’ MCollective remains widely deployed and recently added the ability to write Actions in any language, including Perl, Python, and PHP. Broader language support should bring more developers into the MCollective… er… collective, and existing monitoring and reporting hooks can be easily ported to MCollective. Unlike most alternatives, MCollective does not use SSH for inter-node communication; a message queuing system is used instead. This allows lightweight message passing and asynchronous, parallel processing of messages -- as a result, MCollective is quite performant.  It is Apache-licensed.

As traditional notions of “development” and “operations” blend, programmatic systems administration is becoming more and more common. A solid implementation of one or more of the tools above would help any system be more stable, predictable, and easier to administer.z

Is your team using any of the above tools? What are your thoughts about them? You can find more information on Operations and DevOps right here on InfoQ!

RelatedVendorContent

Deliver quality code quicker with "Go" Agile release management

Continuous Delivery: Anatomy of a Deployment Pipeline

Improving Software Delivery Cycles: Pre-requisites and Inhibitors

Deliver quality code quicker with "Go" Agile release management

App Server Evolution: REST, Cloud, and DevOps Support in Resin 4

Related Sponsor

Go: Agile Release Management Solutions. Go enables predictable, defect-free and timely software releases.

1. Back to top

   Cfengine

   by Aleksey Tsalolikhin

   Nice roundup. I'd like to mention Cfengine as the original open source configuration management tool, for automating system administration of multiple (distributed) servers. Still going strong after 15 years.
   
   Here is an example of multi-node deployment, configuration and integration on Amazon EC2 cloud using Cfengine 3:
   
   www.verticalsysadmin.com/cfengine/casit/
   
   I will be teaching a Cfengine 3 day class at USENIX 2011 conference in Portland on June 17th:
   www.usenix.org/events/fcw11/training/tutonefile...

   Reply