BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles POCs, Scrum, and the Poor Quality of Software Solutions

POCs, Scrum, and the Poor Quality of Software Solutions

Bookmarks

Key Takeaways

  • High quality has three layers: Reliability, Simplicity(elegance), and user-empowering UI. In this article I’ll quantify each of them to make them easier to implement.
  • A POC (Proof of Concept) can prove an idea is doable, it doesn’t validate the implementation is simple – a core requirement for high quality solutions.
  • Introducing a second iteration to the initial POC to validate its simplicity greatly improves the quality. I’m referring to this second iteration as a PONC – Proof of no complexity.
  • Scrum takes the POC and keeps patching it in sprints until it is good-enough for production. Agility often manifests itself as ‘dealing with the urgent’ and leaving out the quality-related work that might not fit into short sprints.
  • MVPs have a great potential for making things worse – don’t commit yourself in the first sprint to technologies you can’t change in the second sprint.

[My writings are personal opinions and don’t represent my company. This article is dedicated to my mentor, Marcel May.]

I don’t take it lightly that I’m implicating POCs and Scrum in producing poor quality software, their popularity is indeed intimidating, but we should be objective and consider all factors in our quest for building high quality software. To be accurate, there is nothing intrinsically flawed in POCs and Scrum that makes them a factor in producing poor quality, but they often create the same motion (side effects) that cause poor quality. 

Precisely because they are not intrinsically flawed, while they are certainly not the only factors, they are among the easiest & most rewarding to solve for when we become aware of these side effects, so it made sense to focus on them today and discuss other factors in future articles. 

This article is divided into two main sections: In section one we take a close look at what quality is and how to quantify it. If defining things is not your cup of tea you can jump straight into section two where we discuss how POCs and Scrum affect quality and how to resolve for that.

For the rest of the article, I’ll refer to ‘high quality’ as ‘Quality’, with capital Q for brevity. 

Section one: Quantifying Quality

Quality is not an endless chase for perfection that can never be attained. It is possible to achieve Quality if we have a way to quantify it. Quality in my view has three layers, we need to quantify each of them individually -

Quality is a measure of reliability

It is generally accepted that quality is the ‘reliability of a product’. ‘Reliability’ though, as we are used to think of in classical science, is the attribute of consistently getting the same results under the same conditions. In this classical view, building a Quality solution means that we should build a product that never fails. Ironically, understanding reliability this way harms Quality instead of achieving it. Aiming to build a product that never fails can only result in extremely complex systems that are hard to maintain causing Quality to degrade over time. The issue with reliability in this classical sense is the false assumption that we control all conditions, while in fact we don’t (hardware failure, network latency, external service throttling…etc.). We need to extend the meaning of reliability to also accommodate for cases when the conditions are not aligned: Quality is not only a measure of how reliable a software product is when it is up & running, but also a measure of how reliable it is when it fails.

To achieve Quality at this layer, you need to train your mind to always think about both cases: the happy path scenario and the worst-case scenario for each component/service of your solution. 

To quantify Quality at this layer, you need to determine your desired Availability, your Recovery Time Objective (RTO) and your Recovery Point Objective (RPO) so you’re clear on the acceptable ranges that your system is committed to meet. 

In short: design for high availability, fail gracefully and withing your acceptable ranges to achieve Quality at layer one.

Quality is a measure of simplicity/elegance

Reliability is the nucleus of Quality, but it is not enough on its own to encompass the entire concept of Quality. Consider this: if you have two machines that have the exact same functionality with exactly the same reliability metrics, the simpler machine is a better-quality machine. 

Quality and performance metrics can coincide, but they are not the same thing. Simpler products are easier to develop, operate, and maintain over time.

Image courtesy to Wikimedia.org

Quality at layer two can be quantified by validating our solution components have a purpose that can be intuitively explained by the requirements. In the second section of this article, we will discuss complexity and I’ll give you practical advice on how to bring simplicity to your projects. Let’s finish up quantifying Quality first, we still have one more layer to discuss:

Quality is a measure of how well the UI represents the functionality of a product

The UI might have a cosmetic layer to it (colors, fonts...etc), it is important that the UI is visually appealing but I’m referring to the UI here from a more holistic perspective – think of the UI as how well the product functionality is represented to the user. Your product might be a set of APIs in which case the UI could be a portal where users can get sample requests/responses and test the endpoints, in some other cases the best UI might be a command line interface, or it can be imbedded where users are already (think Teams/Slack…etc.), or a combination of all of these approaches. It is one thing to create a Quality backend, it is another thing to represent the functionality of the backend in a way that empowers users to consume the functionality in an easy/efficient way. Quality products put the user experience at the center.

To build Quality products at this layer, train your mind to think about your solution from the user perspective, remember that technology is all about serving people at the end of the day.

You can quantify this layer by conducting user testing, aggregate and analyze customer satisfaction drivers and measure your solution against these tangible results. 

Section two: POCs and Scrum — birth and rise of complexity 

Complexity grows exponentially over time, so it is important to mitigate it early on. Take a close look and you will often find that complexity starts with a POC that was prematurely turned into the development pipeline. Scrum short sprints often create an environment most conductive to working reactively to constantly-changing requirements making it hard for developers to prioritize and achieve Quality over the course of the project. As I mentioned in the intro of this article, this motion is not intrinsic to POCs and Scrum, we can resolve these issues once we become aware of them – 

POC – A permission to be messy

POC, Proof of Concept, main objective is to prove a novel idea is conceptually possible. POCs are not concerned with the implementation details. One can ride a horse to the office to prove that it saves time over driving a car but even if this POC is successful it doesn’t mean that riding a horse to the office is the best way or is the only way to replace a car (let alone proofing Simplicity or User Experience or edge cases). Still, this POC has value in that ‘there is a way to get to the office faster than a car’ so we can take this concept to the next level where we look for alternatives and compare outcomes. 

Note that it is counter intuitive to combine the process of ‘finding a solution’ with the process of ‘finding the simplest solution’. It is often the case that our first attempt to solve a problem is not the simplest way to solve that problem. Look at your previous POCs, most likely you will find ‘extra wires’ that can be removed. A successful POC is not alone enough to start development on that POC, we need to try alternatives, ideally several POCs before committing to one of them and move the POC into the next phase.

Proving a ‘novel idea is doable’ is not the only way we use POCs though. Creating a POC is a great way to learn / get a feel for new technologies. The risk in this case is that we mess around with new technologies then we move to the development phase before cleaning up that mess. This is where ‘technical debt’ is often born. 

The solution to this is not to be less messy, quite the opposite actually, we need to be more messy, seriously you should! Often times we can’t clean the mess because we often don’t have enough time to play with new technologies to fully grasp it. Deploying an existing solution from an open-source repo is a great way to get started in a sandbox environment but you should keep working on the sandbox till you are able to map all the outcomes with their causes so you can remove any unneeded pieces when you move out of the sandbox environment. 

This stage of ‘considering alternatives and cleaning up / simplifying’ is sometimes done by developers as part of their work on the POC but under pressure to deliver quickly this phase is often shortened or skipped. This important phase is becoming an imperative given the complexity of most solutions today, so it is time to give it its own dedicated time and incorporate it as part of the project lifecycle. A POC must go through a PONC phase before turning into the development pipeline:

PONC – A guarantee to clean up the mess

A PONC, Proof of No Complexity, is a guardrail for putting simplicity/Quality on track. In this phase you’re looking to simplify and remove any complexity from your POC before turning it to the development pipeline. 

I have written about Complexity and Simplicity in a previous article, essentially Simplicity is achieved when the purpose of each component is easily identifiable in how it ties back to the requirements. Complexity (‘unintuitive complexity’) happens when the implementation is unnecessarily complex vis-à-vis the requirements. The PONC aims to solve for both.

To put a PONC to practice: 

  • Specify your requirements. Write them down and share them with your team, transparency is essential. Don’t solution based on vague requirements as it might make you solve for non-existing issues.
  • Consider alternatives as discussed earlier. Comparing the outcomes, write down the reason(s) you chose this implementation over others*. There are several ways you could implement any solution, especially when the cloud is involved, this early choice of technology is the most important factor in creating Quality software.
  • Remove anything from your POC that is not directly related to the requirements. If you only have basic requirements because you’re still at the start of the project, don’t be afraid to remove everything not related to these requirements even if you’re left with very little. Just stick to good principles (SOLID / 12-factor) and remove everything else – you can always add later when the requirements become clear.
  • Use managed services to reduce infrastructure responsibility. Move right, from IaaS to PaaS to Serverless to SaaS, whenever possible. If you can’t do that, write down and document* the technical blocker that prevents you from doing it, technology keeps evolving and the blocker might be removed in the future so it will be useful to you to know what exactly it is that is blocking you. ‘It is the way we are used to doing it’ is not a good reason for sticking to IaaS.
  • If your solution can benefit from splitting into different services, it is a good time to do it here (E.g. some pieces can move to Serverless, some to PaaS and some others remain as IaaS...etc.).

If you’re re-implementing an existing system, be extra careful not to fall into the ‘second-system effect’ (overengineering because you want to accommodate for every feature you wished the old system had. Second-system effect was first described in the book of Fred Brooks - The Mythical Man-Month). Be clear on what you want to implement so you can maintain simplicity over time (it is always critical to specify your requirements as described above).

(* When I mention you should ‘write down’ or ‘document’ alternatives/outcomes I don’t mean that you should write three page document – one line focused sentence is enough and often better than a long doc. E.g. you can write something like: ‘we chose a Web App because Azure Function doesn’t support Ruby’ – this is enough if in the future you look back at your solution to understand the early decisions and change them if the related reasons become obsolete.)

Even in cases you believe your POC doesn’t need simplification you would still benefit from having a PONC discussion with your team to discuss ‘what if’ we do it this way over that way. When you’re able to articulate why a solution cannot be simplified any further it will give you clarity and confidence that you’re on the right track.

Scrum – A grand design without a design creator? 

Let’s start with the positive – Scrum frames the entire project as a ‘user project’ which is great because it helps us achieve Quality at layer three. ‘User Stories’ are a constant reminder to look at our product from a user perspective.

On the negative side though:

  1. “No sprint zero” approach often results in developers turning their messy POC into an MVP

    The literature on sprint-zero seems to vary. Some scrum textbooks / scrum masters argue against sprint-zero because it is against the basic premise of Scrum where ‘each sprint should produce a value’ (thanks to the Scrum.org team for keeping me honest on their guideline on this), others take a more pragmatic approach and are ok with sprint-zero (as long as you call it a 'Startup sprint'). There is a lengthy online discussion on whether you can/should have a sprint zero or not but regardless of where you stand on this please don't jump into solutioning before you're clear on the requirements and have looked at all potential solutions.

    In fact I believe we can achieve both goals of ‘sprint-zero’ and ‘no-sprint-zero’ by just expanding the way we see ‘value’ (‘value’ here is in the context of the Scrum rule that ‘each sprint should produce a ‘value’):

    ‘Value’ can be an MVP, of course. But ‘value’ can also be a visual prototype (think Sketch or InVision and similar tools). To my mind, anything that you can show to the product owner to collect their feedback on should be enough ‘value’ for sprint one. You would be surprised how much good you can bring to your scrum team by showing the product owner your early work, capturing their feedback early on will not only keep you close to their vision but also will allow the dev team to use the right business terminology instead of each using their own that might mean different things to different developers.

    Reducing ‘value’ to just mean an MVP or a piece of functional product can be detrimental to Quality because it can force developers to commit to technologies before they have a chance to grasp all the requirements. When the requirements become clear at subsequent sprints it is often too hard to change these early technologies resulting in developers adding more and more workarounds to satisfy the requirements instead of natively solving them by matching them with the right technology. Starting with a lacking MVP then patching it in sprints can only result in a poor quality product. If Quality was a person, he/she would be horrified at the idea of Sprint-one turning a messy POC into an MVP that ends up being deployed to production.

    (I want to take this opportunity to invite the Scrum guideline creators to clarify ‘value’ in their Scrum guides to also include visual prototypes and architectural artifacts that can be shared with the product owner for collecting their feedback. This will make it easier on developers wanting to implement Scrum the right way.)

    Lastly, here are few things that might help you navigate the sprint-zero depending on your project/team maturity and timeline for project delivery –

    •  Design Sprints can give you great benefits in some situations. Although their results are not necessarily releasable (as desired by Scrum) they can provide a lot of value in some cases.
    • If you’re in a situation where you have to build an MVP before you finished the architecture discussions/POCs/PONC, consider creating an MVP with (fully or partially) mocked backend (e.g. use Azure APIM), this will allow you to change the backend technologies while at the same time give product owner / testers a chance to put their hands on something concrete.
  2. Scrum's focus on layer three creates a high risk of violating Quality at layers one and two
    Quality at layer one and two (think of disaster recovery, maintaining simplicity…etc) like any other non-functional requirements (NFRs) are usually deprioritized under pressure to deliver on the urgent items. It is a good practice to put NFRs on the product backlog (and maybe incorporate some of them in your Definition of Done if possible), this will certainly bring transparency and will greatly facilitate the discussion of these items within your scrum team. You remain with one challenge, you need to prioritize these items to get them implemented. This is easier said than done, especially in an agile environment like scrum.

    Agility’s chief goal is to free you from sticking to a stiff project plan that prevents you from responding to changes, this is great because it guarantees you’re not drifting far from what the stakeholders envisioned for their product. However, changes are inevitable, and agility main goals will diminish if you don’t respond to these frequent changes. The side effect is that you will often prioritize these changes over the Quality-related work. You might not notice the reactive nature of Scrum when you look at it from within each sprint individually, but if you zoom out to see the entire project, you will see that you’re in a reactive environment where prioritizing NFRs needs special attention to materialize. No two scrum teams are the same, so I don’t know if there are universal rules on how to prioritize Quality at all its layers but now that you’re aware of this motion I trust you’re the best person to put solutions for prioritizing Quality in your projects.

Conclusion

In this article, I talked about the three layers of Quality and how to quantify each layer. Poor quality starts early on, a POC should be followed by a PONC before turning it into the development pipeline. Scrum helps us achieve Quality at layer three, but we need to have a strategy and pay special attention to prioritizing Quality at layers one and two.

About the Author

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Fairly misinformed about the entire process.

    by Daniel Smith,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    For one, scrum expects a done increment every sprint. That means a piece of work done to an agreed point (quality included) where it is in a releasable state (ie it does not need further work at the end of the sprint). Scrum in no way encourages POC's or just continuing to build on a bad state, it does encourage quality work that is releasable (quality etc) from the outset.

  • Re: Fairly misinformed about the entire process.

    by Alaa Tadmori,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Hello Daniel. You're describing the theory of Scrum. The article is describing what often happens when this theory is put to practice, especially in projects with tight deadlines. The goal of the article is to shed light on some negative motions so we can identify and solve for them in a way that keeps us aligned with the theory of Scrum.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT