“An agile enterprise is able to anticipate and respond swiftly to changes in the marketplace” says Scott Ambler. Today Scott gave a presentation about the disciplined agile enterprise at the OOP conference in Munich, Germany, in which he talked about how organizations can evolve from doing agile software development to become a disciplined agile enterprise.
InfoQ is covering the OOP 2015 conference with live news write-ups and Q&As and will be publishing articles from conference speakers. Earlier InfoQ published a Q&A with Kevlin Henney and Frank Buschmann about using the "Worse is Better" concept with agile and lean and a Q&A with Patrick Kua on technical leadership for agile teams.
InfoQ interviewed Scott about the reasons why agile projects are failing, how to increase budgets for building new systems, disciplined DevOps, harmonizing agile and lean, and on coaching for enterprise agility.
InfoQ: According to the DDJ Sept 2012 State of IT Union Survey 55% of the organizations have some agile projects that failed. Do you know the main reasons why agile projects are failing?
Scott: There are many reasons why organizations will fail with agile. Many times the organization doesn’t invest sufficiently in agile training and coaching. A common mistake is to send people on a Scrum certification course and then to tell them to be agile. Although I’m sure they’ll learn some interesting ideas in the Scrum course, in practice Scrum proves to be a very small part of the overall strategy. To be successful you need to adopt Full Agile Delivery Lifecycles (unfortunately Scrum focuses on Construction and has little to say about project initiation/inception or about deployment/release). You also need to address all aspects of software development, including testing, programming, architecture, analysis, management, and so on. Scrum just covers team leadership/management and requirements change management, you need to adopt techniques from other methods such as Extreme Programming, Agile Modeling, Kanban, Agile Data, Unified Process and others to cover the full range of what it means to be agile. Luckily the Disciplined Agile Delivery (DAD) process framework has done this work for you already, but you’d need to know enough to actually start with DAD. Unfortunately many organizations get fooled by the overwhelming marketing rhetoric around Scrum and end up learning the hard way.
Another common mistake is to get fooled by the developers who are simply hacking and claiming to be agile. Many of the “agile failures” are really a combination of immature developers being overseen by managers who don’t know enough about agile to realize that they’re not following an agile approach.
We’ve also seen teams that adopt the wrong flavor of agile or lean at first, only to succeed later by adopting the right one. This is why the DAD framework supports several Delivery Lifecycles - A Scrum-based one, a Lean lifecycle, a continuous delivery lifecycle, and an exploratory/lean startup-based lifecycle - so that you have a choice. Prescriptive methods such as Scrum only support one lifecycle and as a result often prove to be too inflexible for enterprise adoptions of agile.
InfoQ: You mentioned that typically 65-80% of the IT budget is spend on keeping existing systems up and running and only 15-25% on building new systems, What can enterprises do if they want to be invest a bigger part of the budget on building new systems?
Scott: These sorts of ratios are all too common in today’s organizations. In part this reflects the fact that some organizations have decades worth of systems to operate and support in production, and in part it reflects a lack of maturity when it comes to supporting common infrastructures, common development guidelines, and helping teams to follow shared technical and business roadmaps. The DAD framework promotes the idea that development teams should be enterprise aware, that they should work closely with other teams so that they do what is best for their organization, not just what is immediately convenient for their team. Enterprise awareness, and the desire to streamline the way that your entire IT department works, are important aspects of moving to an Agile IT organization (Agility at Scale).
InfoQ: Can you give some suggestions how enterprises can deploy DevOps to streamline IT development and operation?
Scott: Funny you should ask that. I’m currently writing a detailed article on this very subject which will soon be available. Where “standard” DevOps focuses on streamlining IT development and IT Operations, Disciplined DevOps strives to streamline IT Development, IT Operations, Business Operations, Release Management, Data Management, and Enterprise Architecture.
There are many DevOps strategies that teams can adopt, including automated dashboards (for both development and operational metrics), integrated change management, integrated configuration management, continuous delivery, feature toggles, automated regression testing, continuous integration, integrated deployment planning, technical and business roadmaps, common development guidelines (coding, data, security, …), collaborative architecture, collaborative data management, and many more. We’re still in very early days when it comes to DevOps in most organizations.
InfoQ: Can you give some examples how enterprises can harmonize agile and lean? What would be the benefits of doing this?
Scott: We run into this all the time. The DAD framework was arguably misnamed because it weaves lean as well as agile principles and practices into a coherent whole. It is quite common to find organizations that have some agile teams working side-by-side with some lean/kanban teams, which is one of the reasons why the DAD framework includes several lifecycles as I mentioned earlier. Our philosophy is that teams should choose the right approach for them, and thereby increase their ability to produce value for your organization. The challenge is that anyone who need to interact with several development teams, such as your governance people, enterprise architects, portfolio managers, product managers, data management, and many other people need to be sufficiently flexible to work with each team in a manner than reflects the way the team works. For example, your enterprise architects will need to work with agile and lean teams in a evolutionary and collaborative manner, whereas they will likely work with traditional teams in a more documentation-driven manner.
InfoQ: You stated that "Coaches are very easy to find, good coaches experienced in enterprise agile are not". Can you elaborate on this?
Scott: The challenge that we see is pretty much anyone and everyone is hanging up a “Agile Coach” shingle these days. Apparently a few years experience on an agile team makes people think that they’re qualified to be a coach. This MIGHT be true at the project level, but it certainly isn’t true at the enterprise level. Enterprise coaches need deep experience in IT, not just in agile software development. They need to understand, and be experienced with, activities such as data management, enterprise architecture, IT governance, operations, support, portfolio management, and many other critical aspects of IT if they are going to help an organize move to an enterprise agile strategy. These various teams have different priorities, different cultures, and different ways of working and a good enterprise coach needs to understand that. To put things in perspective, my company has several enterprise agile coaches on staff, including myself. I am currently in my late 40s and I am the youngest consultant in the group.
InfoQ: If enterprises want to increase their agility, is there some advice that you can give them to take a first step?
Scott: Best advice that I can give is to stop looking for easy answers. Software development is hard and IT is even harder. Agile IT enables the agile enterprise, and moving to an agile IT strategy will take many years of hard work. Our Agility at Scale provides a pretty good overview of what organizations are looking at.