Understanding Production with DevOps Archeology

| by Manuel Pais Follow 9 Followers on Jun 14, 2018. Estimated reading time: 1 minute |

Lee Fox, cloud architect at Infor, spoke last month (video), at the Continuous Lifecycle London conference, about tools and methods to help make sense of today's complex systems and infrastructure (PDF of slides). Fox calls this DevOps archeology, in analogy with the area of software archeology. The core idea is to analyze and visualize (application and infrastructure) code to extract insights on the behavior of production systems and the context and rationale for decisions that were made in the past - often by people no longer working in the organization.

Fox mentioned different categories of tools that can help digging into today's highly interconnected systems, such as network sniffing (for example Charles Proxy), monitoring (for example Amazon CloudWatch), log aggregation (for example the ELK stack), testing and auditing/analysis (for example AWS Trusted Advisor). Regardless of which tools to use, Fox recommends following the scientific approach of hypothesizing about the answer to a given question, running an experiment (analyzing the code), and validating if the results corroborate the hypothesis or not.

Artifacts from the tools mentioned provide a wealth of data to analyze, but sometimes a clear visualization makes things more evident. Fox highlighted how tools like Gource - to visualize codebase evolution - and Logstalgia - to visualize traffic to a web server - can help surface relevant events. Digging into those events can then raise questions such as "why was there an explosion in number of source files around this date?" or "why were virtual machines sized this way?".

A screenshot from Logstalgia

In order to perform this kind of analysis, it is fundamental to have everything in version control (code, configuration, scripts, etc). Fox recommends creating isolated branches for "digging" into the code and running a given experiment, without disturbing the regular integration of development changes.

In terms of integrating DevOps archeology in the value stream activities, Fox strongly recommends performing this kind of analysis at the start of any engagement, but also as a continuous learning activity about the systems in production. Fox finalized by admitting that to fully understand DevOps history inside an organization we need to consider the people and processes side, on top of the tooling mentioned. That is the focus of "DevOps anthropology", his next venture in this space.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread


Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you