“A leader is a coach, not a judge” –W Edward Deming was not teaching a ScrumMaster course when he said this, but needless to say - the statement will resonate with every ScrumMaster worth the name.
Deming was one of a select group of thought leaders who have shaped modern management over the last century. He is best known for the impact he had on Japanese businesses with his teachings on design, manufacturing, sales and quality. Japan adopted Deming’s (an American) teachings before the US did, and derived huge productivity and quality gains. His teachings have been credited for enabling Japanese companies to dominate markets around the world with some great products. In appreciation of the work Deming did, the Japanese Union of Scientists and Engineers (JUSE) named the country’s national quality award after him. The Deming Prize is the world’s oldest and among the most prestigious awards for industry. It was not until much later that Deming’s teachings began to get adopted in the US. In his later years, he helped executives at companies like Ford Motors and Xerox improve their management styles.
What does Deming have to do with agile software development? After all, his teachings were focused on manufacturing efficiencies – and some of the weaknesses of a traditional, waterfall software development methodology have been a direct consequence of trying to apply traditional engineering processes to building software. As practitioners of agile methods – we certainly want to be cognizant of the different nature of software projects and avoid the pitfalls of the traditional approach. But the management philosophy in these teachings transcends specific engineering disciplines. There is a wealth of wisdom here that if applied properly will help increase the effectiveness of agile practices.
Deming’s most famous teachings are a collection of management advices that he published in the book “Out of the Crisis” in 1986. These are known as “Deming’s 14 Points”. The relevance of these 14 points to agile practices cannot be overstated. As agile processes mature, it is critical that we as practitioners stay true to the spirit of the agile manifesto, and not be carried away by the tasks defined in the many variants of agile methods. One sure way to do this is to validate our decisions in running a project with Deming’s 14 points. We can use these teachings as an undercurrent that drives the project. In the next few sections, this article will discuss the relevance of each of the points and demonstrate how they can be used as a guide in an agile project.
Point 1: Create constancy of purpose toward improvement of product and service, with the aim to become competitive and to stay in business, and to provide jobs.
Software developers in many organizations don’t concern themselves with strategic direction of the business. The superficial practice of agile often exacerbates this practice. While practicing short iterations with near term deliverables, teams often make the mistake of ignoring anything beyond the current iteration – or at best, the current project.
Deming’s teaching not only guides the business to focus on its purpose, for software teams, it emphasizes an often ignored best practice – know your project’s real purpose and stay true to it.
Software products are developed to fulfill some specific business goals. These goals are derived from the strategic direction of the company. The corporate strategy is supported by a marketing strategy and implemented by the product portfolio plan. It is important to be aware of this strategic view (figure 1) while executing individual stories in projects.
To effectively build a product, the project team needs to be aware of the overall company strategy, along with the marketing and product strategy that has necessitated the project.
A specific iteration may have 3 or 4 user stories to be developed, but while developing these stories, it is critical to keep in mind that the whole is always greater than the sum of the parts. If we focus only on the individual stories, without considering how these stories impact the rest of the product and the portfolio that the products is part of, we will probably not do justice to the story itself.
To give a very simplistic example- say an iteration includes a story which requires developing a web service that captures data feeds from an external source. The story may not say anything about the use of the captured data. In this scenario, unless the developer has an understanding of how the data is to be used by downstream products, she will not be able to properly design the persistence strategy for the data. If the data needs to go through further transformations, it will need to be treated in a certain way – and if it is used only for reporting purposes, it may require a completely different approach.
This level of understanding will only come from spending the time to learn the strategic direction of the product, its intended use, target audience and shelf life.
Figure 1: Strategic view of stories
This teaching from Deming also underscores the need for effective communication. Often business stakeholders don’t spend the time to explain the product’s purpose, and the engineers don’t bother finding out. Because of this teams often end up working on cross purposes, resulting in poor quality products that don’t serve the real need of the organization.
A couple of best practices that can be helpful in this regard are:
- Develop a high level project plan upfront that covers the entire project – this may seem to go against agile teachings, but it actually does not. Agile principles advice not to work out detailed plans at a task level upfront – but it does not discourage planning in any way. High level plans that lay out milestones and list all major product features help provide the necessary context to individual stories.
- Document the product vision and communicate it constantly to all stakeholders. The vision document need not be very detailed – but it should contain enough information that clearly explains the strategy behind the product, its intended use and expectation of sponsors. The team then needs to make it a point to regularly go back to the vision document throughout the project and make any necessary course correction to stay in sync with the vision.
Point 2: Adopt the new philosophy. We are in a new economic age. Western management must awaken to the challenge, must learn their responsibilities, and take on leadership for change.
Deming’s message here is primarily for the leadership team. He urges management to adopt change and incorporate it into the organization’s life. The change Deming refers to here stems from the practice of total quality management, which requires that management leads from the front, empowers workers and moves the organization towards achieving its competitive goals in an ever changing economic environment.
This message of organizational change cannot be more relevant in an agile setting. The concepts of agile require transformation that is not restricted to the development team. Successful agile implementations need executive champions, stakeholder buy-in and structural change in the organization – and these can only happen through executive sponsorship. Many agile teams have failed precisely because of the lack of executive support and have gone back to waterfall.
Agile introduces a new way of thinking about technology projects.. It requires adopting new product marketing strategies, changing product release methods and changing budgeting processes. It also requires operational adjustments in customer service and sales. Agile developers can put a great team together and execute flawlessly – but the end results still won’t be nearly as good if product planning is not tailored to feed incremental enhancements to the team or the marketing message is not adjusted to incremental releases. This cohesive effort can only be achieved with a high performance cross-functional team that goes across department boundaries and has executive support.
Like it or not, this process takes time and patience. This is probably the most difficult change to implement in an organization. Regardless, this is a critical component for success and needs to be done.
One way of expediting this is to create a role of an agile evangelist in the organization. This role needs to be filled by an accomplished agilist who can effectively communicate with senior management and mentor the various stakeholders.
Point 3: Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.
For most software projects, quality is an afterthought. It is something that is added to the process towards the end of development, and its responsibility is thought to be solely of the testing team.
Agile methodologies have a fundamentally different approach to quality – an approach that is inherent in this teaching from Deming. Quality cannot be achieved by testers inspecting work done by developers. Quality can only be achieved if developers, business analysts, architects and testers – all take measures to ensure that it is part of every task they perform. Testers need to focus on testing the behavior of the product in the context of business needs. They should not need to test if a unit of code behaves as it is supposed to. Every developer needs to internalize the need to produce high quality code; code that does what it is designed to do.
Agile methods have some very elegant practices that facilitate building quality into the product. The practice of test driven development – and continuous integration, if implemented properly, reduces the chance of low quality products being released for QA testing. Implementing these processes may sometimes seem like an overhead, and not all development teams will have the infrastructure or the resources to implement these fully.
But quality ingrained in every task performed is a goal that will serve any team well.
Another critical factor directly impacting quality is strong participation from all stakeholders. A core agile principle requires that ‘business people and developers must work together daily throughout the project’ (http://agilemanifesto.org/principles.html)
Projects that don’t have active participation from subject matter experts, user groups, marketing, sales and other relevant groups within the business typically fail to build quality in the product upfront. These projects then depend on inspection after the fact resulting in poor quality end products.
As we have seen in the 3 points discussed in the preceding sections, Deming’s teachings are strongly aligned with the principles of agile projects. The teachings can be used by new or experienced agile practitioners as a succinct set of guidelines to help them achieve agility. We will continue to discuss the remaining 11 points and their relevance to the agile world in forthcoming issues.