Enhancing Your "Definition of Done" Can Improve Your Minimum Viable Architecture

A Definition of Done (DoD) is a description of the criteria that a development team and its stakeholders have established to determine whether a software product is releasable (for a more complete explanation of the Definition of Done in an agile context, see this blog post). The DoD helps to ensure that all stakeholders have a shared understanding of what minimum bar a product increment must achieve before it can be released. The DoD usually focuses on functional aspects of quality, but teams can strengthen the quality and sustainability of their products if they expand their DoD to include architectural considerations.

Architecture and the Definition of Done

Typical contents of a DoD include criteria such as:

• The code must pass code review/inspection
• The code must pass all unit tests
• The code must pass all functional tests
• The product must be defect-free
• The build must be deployed for testing on its target platforms
• The code must be documented.

Extending a DoD with architectural concerns means strengthening the DoD with criteria that assess the sustainability of the solution over time. Examples of this include:

• Assessments of the code’s maintainability and extensibility
• Assessments of the security of the product
• Assessments of the scalability of the product
• Assessments of the performance of the product
• Assessments of the resiliency of the product
• Assessments of the amount of technical debt (TD) incurred during the development of the product, and estimates for "repaying" that technical debt

By extending their DoD with architectural concerns, a team is able to evaluate the sustainability of their product over time every time they evaluate whether their product can be released.

Software architecture, agility, and the DoD

In a series of prior articles, we introduced the Minimum Viable Architecture (MVA) concept, which provides the architectural foundation for a Minimum Viable Product (MVP). Many people think of an MVP as something that is useful only in the early stages of product development.

We don’t think of  MVPs and MVAs as "one and done", but instead as a series of incremental deliveries. The product evolves MVP by MVP. Each MVP should ideally increase customer satisfaction with the product, while each MVA should increase the system's quality while supporting the MVP delivery (see Figure 1). When a team uses an agile approach, every release is an incremental MVP that helps a team to evaluate the value of that release.

Figure 1: A definition of an MVA

An MVP is an implicit promise to customers that what they are experiencing will continue to be valuable for the life of the product. It is not a "throw-away" prototype to evaluate a concept; the "product" part of MVP implies that a customer who finds the MVP valuable can reasonably expect that they will be able to continue to use the product in the future.

Because of this implicit promise of support over time, every MVP has a supporting MVA that provides an essential architectural foundation to ensure that the MVP will be sustainably supportable over time in a sustainable manner. While the typical DoD focuses on functionality, i.e. the MVP, extending the DoD to evaluate architectural concerns helps a team to make sure that the MVA is also "fit for purpose."  When extended to consider architectural concerns, the DoD can help a team evaluate whether its MVA is "good enough." 

Just as the MVP can only be evaluated by delivering running code to real users or customers and getting their feedback, the MVA can only be assessed by stringent testing and evaluation of executing code. The only way to validate the assumptions made by an architecture is to build it and subject the critical parts of the system to tests. Not the whole system, but enough of it that critical architectural decisions can be evaluated empirically. This is the concept behind the MVA.

In an agile approach, time is not on your side

The challenge that teams who work in short cycles using agile approaches face is that they have almost no chance of really evaluating the quality and sustainability of their architecture without automating most of the evaluation work. Most teams are challenged to simply develop a stable, potentially valuable MVP. Making sure the MVP is sustainable by building an MVA is yet more work.

Without automation, teams simply cannot find the time to develop the MVP/MVA and evaluate whether it meets the DoD. Adding more people does not help, as doing so will drag down the effectiveness of the team. Handing the evaluation off to another team is similarly ineffective as it also reduces team effectiveness and slows the receipt of important feedback.

The most effective way to automate the DoD is to build it into the team’s automated continuous delivery (CD) pipeline, supplemented by longer-running quality tests that help to evaluate the fitness for purpose of the MVA. This approach is sometimes referred to as using fitness functions. Doing so has several benefits:

• Developers obtain immediate feedback about the quality of their work
• The entire team benefits by not having to work around changes that are not yet "fully baked".
• Parts of the system that cannot be evaluated when code is delivered into the code repository are checked frequently to catch architectural degradation.

In practical terms, this often includes adding the following kinds of tools and evaluation steps to the CD pipeline:

• Code scanning tools. Tools like lint and its successors have been around for a long time. Though many developers use IDEs that catch common coding errors, having a way to check for common coding errors, the existence of comments, usage of style conventions, and many other conditions that can be caught by static code analysis is a useful starting point for automating some basic architectural assessment of the system. For example, some types of security vulnerabilities can also be caught by code-scanning tools, and modularity and code complexity problems can be flagged. The usual argument against code scanning is that, if the rules are not tuned effectively it can generate more noise than signal. When it does, developers ignore it and it becomes ineffective.
• Build tools. Making sure code can be built into an executable using agreed-upon standard architectural components and environment settings uncovers subtle problems that often fall under the "it works fine on my machine" category of bugs. When developers use different versions of components, or new components not agreed upon by the rest of the team, they introduce errors caused by configurations that are hard to isolate. Sometimes old versions of components have known security vulnerabilities so building with patched versions helps to eliminate security vulnerabilities.
• Automated provisioning and deployment tools. Once the code is built into an executable, it needs to be deployed into a standard testing environment. Decisions about standard configurations are important architectural decisions; without them, code can fail in unpredictable ways. When these environments are hand-built by developers, their configurations can "drift" from those of the eventual production environment, resulting in another class of "it works fine in the testing environment" kinds of errors. We’ve also noticed that when it’s not easy to create a fresh testing environment, such as when testing environments have to be manually created, people will reuse testing environments. Over time, this can cause the testing environment’s state to drift from that of the intended production environment. Similarly, when someone has to manually deploy code they can forget things, or even just do things in a different order, so that the testing environment is subtly but significantly different from the production environment.
• Unit testing tools. At its most fundamental level, unit testing makes sure that APIs work the way that the team has agreed they should. While automated builds catch degradation in interfaces, unit testing makes sure the agreed-upon behavior is still provided. Contract testing approaches provide a useful way to evaluate when applications break API commitments.

Incorporating these checks into the CD pipeline provides immediate feedback on the architectural fitness of the system, but to do so the checks need to run quickly or developers will rebel. The architectural checks in the CD pipeline represent only the first line of defense. Checks that provide deeper inspection should be run as automated background tasks, perhaps run overnight, or periodically in the background. These include:

• API-based testing of functional requirements. Agile development cycles are too short to do meaningful manual testing. As a result, everything, including user interfaces, needs to have an API that can be driven by automated tests. Functional testing isn’t architectural, per se, but giving everything an API is useful for automating integration and system testing in a scalable way.
• API-based scalability, throughput, and performance testing. These tests include things like scaling up the number of user instances, or process instances, to see what happens to the system under load. Even when the back-end consists largely of microservices that are dynamically replicated by cloud infrastructure, you still need to be able to easily scale the number of clients.
• Automated reliability testing. Approaches like Netflix’s "chaos monkey" help to evaluate how a system will respond to various kinds of failures, which in turn helps a team to develop a resilient architecture.

Other useful tools and manual techniques include:

• Test data management. Creating, maintaining, randomizing, and refreshing test data is complex and error-prone. Because it’s hard, people tend not to do it; they copy production data, leading to security vulnerabilities, or they use small manually-created test data that does not expose real-world edge cases or data volumes. The result is that important quality issues can remain undiscovered, including important behavioral defects, unhandled out-of-bounds conditions, stack overflows, race conditions, and security errors, to name a few
• Code and architectural reviews. Manual inspections for code quality, architectural integrity, reliability, and maintainability can be valuable, but they usually can’t be release-blocking if the organization wants to deliver on short cycles. The issues that they find are real, and need to be addressed, but if the organization wants to obtain fast customer feedback, manual reviews have to be focused on important but non-blocking quality problems. Manual inspections simply cannot respond to the speed and volume of change.
• Ethical hacking and simulated attacks. We live in a world in which vulnerabilities are constantly tested. Known vulnerability testing needs to be automated, as an extension of basic vulnerability checking in code scans, but there is still an important role for having skilled, creative, but friendly hackers constantly trying to infiltrate the system. The vulnerabilities they find should be folded into the automated testing described above.

Can the Definition of Done be overridden?

Strictly speaking, no. If the team agrees that the DoD criteria really represent the minimum quality bar that it is willing to accept for a released product increment (MVP), they should not lower their standards for any reason. No one is forcing them to adopt their DoD, it’s their decision, but if it means anything they should not abandon it because it’s inconvenient.

This means that a team should thoughtfully discuss what releasable means to them and to their customers. While their DoD might be absolute, they might also want to have a separate category of tests they run to better inform their release decisions and their architectural decisions. If they are willing to release an MVP to obtain customer feedback they might be willing, especially early in the product’s lifecycle, to sacrifice scalability or maintainability, recognizing that they are consciously incurring technical debt. In making that decision, however, they need to understand that if they make a decision that renders the product unsustainable, they may inadvertently kill the product.

Is the architecture ever done?

With each MVP delivery, there are several possible outcomes:

1. The MVP is successful and we believe that the MVA doesn’t need to change.
2. The MVP is successful but the MVA is not sustainable. This outcome could follow the previous one after a period of time, as the MVA may decay over time, even if the MVP does not change.
3. The MVP is mostly successful as its functionality meets a majority of the business stakeholders’ needs, but the MVA is not, as its Quality Attribute Requirements (QARs) are not fully met. The MVA needs improvement, and technical debt may accrue as a result.
4. The MVP is partially but not wholly successful, but it can be fixed. As a result, the MVA needs to change, too. Again, technical debt may accrue as a result.
5. The MVP isn’t successful and so the MVA doesn’t matter

We can group these five potential outcomes into the following four scenarios to help us evaluate whether we are really "done" by asking ourselves a few key questions as follows:

• Scenario 1 (outcome 1): In this scenario, we may believe that we are "done" since both MVA and MVP appear to be successful. However, are we really done? What about the sustainability of the MVA? Do we really believe that the MVA won’t change over time and that technical debt (TD) won't start accruing as a result of these changes?
• Scenario 2: (outcome 2): Here, we believe that we are "done" with the MVP, but we have doubts about the long-term sustainability of the MVA. How can we confirm these doubts and prove that the MVA isn’t sustainable? Did we incur an excessive amount of TD while delivering the MVP? Did we test that key QARs such as scalability, performance, and resiliency will be met over a reasonable period of time? Can we estimate the effort necessary to "repay" some of the TD we have incurred so far?
• Scenario 3 (outcomes 3 and 4) - In this scenario, the MVP is mostly successful but we believe that the MVA isn’t sustainable since we need to make some significant architectural changes either immediately or in the near future. In other words, we know that we are not "done" with the architecture. How can we assess the sustainability of the MVA? Did we test whether the system is scalable and resilient? How will its performance change over time? How much TD was incurred in the MVP delivery?
• Scenario 4 (outcome 5) is the only one where we can say that "we are done" - but obviously not in a good way. This means going back to the drawing board, usually because the organization’s understanding of customer needs was significantly lacking.

In summary, architectural criteria in a DoD are ephemeral in nature. MVAs are never really "done" as long as MVPs remain in use. As we noted in an earlier article, teams have to resist two temptations regarding their MVA: the first is ignoring its long-term value altogether and focusing only on quickly delivering functional product capabilities using a "throw-away" MVA. The second temptation they must resist is over-architecting the MVA to solve problems they may never encounter. This latter temptation bedevils traditional teams, who are under the illusion that they have the luxury of time, while the first temptation usually afflicts agile teams, who never seem to have enough time for the work they take on.

Conclusion

The Definition of Done helps agile teams make release decisions and guide their work. Most DoDs focus on functional completeness, and in so doing risk releasing products that may become unsustainable over time. Adding architectural considerations to their DoD helps teams correct this problem.

The problem, for most teams, is that they barely have enough time to evaluate a functional DoD, let alone consider architectural concerns. They cannot, realistically, reduce scope to make room for this work and risk unsatisfied stakeholders. Their only realistic solution is to automate as much of their DoD, including architectural concerns, to give themselves room to handle architectural concerns as they come up.

In expanding their DoD with architectural concerns, the concept of a Minimum Viable Architecture (MVA) helps them limit the scope of the architectural work to that which is only absolutely necessary to support their latest release, which can be thought of as an evolution of their Minimum Viable Product (MVP).

Evaluating the MVA (and functional aspects of the release) using a largely automated DoD provides teams with concrete empirical evidence that their architecture is fit for purpose, something that purely manual inspections can never do. Manual inspection still has a role, but more in developing new ideas and novel approaches to solving problems.

Modern software architecture has evolved into a continual flow of decisions that are revisited continuously. A Definition of Done, extended with architectural concerns, helps teams to continually examine and validate, or reject, their architectural decisions, preventing unhealthy architectural technical debt from accumulating.