BT

FOSDEM Configuration Management: Open Source Infrastructure

| by Carlos Sanchez Follow 0 Followers on Feb 03, 2015. Estimated reading time: 3 minutes |

Spencer Krum and Elizabeth K. Joseph shared their experience both using and providing the public infrastructure used by OpenStack at the configuration management developer room at FOSDEM.

Consuming Open Source Infrastructure

Spencer Krum, cloud engineer at HP, works on the HP distribution of OpenStack that uses the upstream Puppet based infrastructure created by the OpenStack.org team. HP applies their own patches to OpenStack and then test them internally, to build their own OpenStack distribution.

We benefit from consuming the infrastructure because we have confidence that the architecture is viable. Any departures you take from upstream is technical debt.

In HP case, the upstream OpenStack.org team used two OpenStack clouds, while downstream at HP they used one OpenStack cloud across two datacenters, something that caused networking issues amongst other problems. For downstream the goal becomes deciding what to replicate, while for upstream it is decoupling components from each other so they can be easily consumed downstream.

At HP they contributed changes upstream, ensuring the contributions were small changes or conditionals to consider specific cases that could be customized downstream, both for practical reasons, because big patches are harder to process and makes life harder for the upstream committers, and for social reasons, creating a trust relationship between committers and contributors.

They had to face issues with different versions of Puppet between the upstream and downstream repositories, usage of old modules, like the Puppet Apache one, and a monolithic Puppet repository. There was an effort to migrate from Puppet 2.7 to 3.7, splitting the data into separate git repositories and the monolithic git repository into repositories per module, using Puppet Blacksmith to release modules to the Puppet Forge, and running integration testing against proposed changes.

The current plans for the upstream OpenStack repository includes using semantic versioning for modules, and using git clone to fetch module dependencies to avoid transitive dependency issues. In the downstream repository at HP they plan to run everything in OpenStack to test their own environment, update to Puppet 3 and deploy more testing.

Fully Public Puppet

Elizabeth K. Joseph, automation and tools engineer at HP, works on the OpenStack.org CI team, and talked about the project public infrastructure.

She outlined the benefits of having public infrastructure:

  • Provides examples.
  • Encourages better practices.
  • Allows others in your organization to suggest changes.
  • Sharing with the community is nice.

OpenStack has a policy to use only open source tools, to avoid conflicts between commercial companies. There are other projects that have open sourced their infrastructure configuration and provide a public view, Debian, Mozilla or Jenkins for instance.

In the specific case of Puppet, she recommends to:

  • Leverage existing modules, or write new ones with the intent of sharing.
  • Split existing system configuration and Puppet configuration.
  • Split out non-sensitive custom configurations into a different module.
  • Use Hiera for sensitive data.
  • Document modules, linking to the source, explaining how to use the configuration and how to contribute with commits and bug reports.
  • Add a license to configuration files. This is important for reusing them.

OpenStack uses pic, a web interface to PuppetDB aiming to replace the reporting functionality of Puppet Dashboard. It allows them to give visibility to developers on whether their patches are applied in servers, or getting the status of the server, without having to ask the infra team. It empowers developers to have the visibility over the infrastructure.

On the question about using Puppet and not something else, se answered:

We use Puppet because we are a small team and the person trying Puppet and Chef got Puppet running first. If we were to start from scratch we would probably use Ansible to avoid having agents everywhere.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT