BT

Virtual Machine Test Harness (VMTH): Test Your Configuration Management

by Chris Brown on May 20, 2011 |

A Google search for “devops” yields tons of interesting postings, tools, and even a couple of manifestos. Refine the search further to “devops quality” and the results become less direct; “devops testing” simply doesn’t exist according to Google.

That said, things are starting to change.

In late April, Greg Retkowski released Virtual Machine Test Harness (VMTH), for unit-testing infrastructure automation. Application developers are used to seeing a “green bar” or other reports that indicate that the changes they are making have been automatically tested and will not break predefined scenarios.  VMTH does this for your configuration management (Puppet/Chef/CFEngine) setup, automatically building out virtual machines based on a configuration management setup, and then running tests against those machines.

As referred to in the source code, tests are set up in YAML. They are composed of directives to run uploads and downloads to the test VM, and of course to set up and run the tests. The desired tests can be specified in a plain-text scenarios file or passed in from the command line.

For example, one could set up a scenario file to kickstart a new machine, deploy and configure a database server, load-balancer, and web server, then ask VMTH to verify that the configuration works:

VM Test Harness Results....
Elapsed time to complete testing: 06:08:14
------------------+------------+------------+----------------+
 Scenario         | Apply      | Test       | Execution Time |
------------------+------------+------------+----------------+
 0nulltest        |     Passed |     Passed |       00:22:37 |
 dbserver         |     Passed |     Passed |       00:09:29 |
 loadbalancer     |     Passed |     Passed |       00:04:04 |
 webserver        |     Passed |     Passed |       00:04:03 |
 ...

The test shows that the target machine’s services are configured correctly and (in this case) indeed actually up and running. While the tests take a long time -- several hours for a full run -- no human intervention is required, and the outcome is rock-solid confidence in your configuration. It’s roughly the DevOps equivalent of GUI testing tools like Squish and Selenium.

VMTH currently only works in a QEMU+KVM environment, but Greg points out that the VM-manager part of the software is abstracted and he’s “hoping to hear from some folks using xen or vmware to see their experiences.” VMTH is Apache licensed.

Increased test automation is probably the most significant change in the software quality and testing arena over the past few years. While developing for DevOps is simply different than developing a typical application -- infrastructure is hard to mock out, and cycle times are much longer -- VMTH shows that many application development techniques and tools are transferrable to DevOps.

You can find more information on Operations and Testing, right here on InfoQ!

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT