The Management View of Agile - Unaware or Unwilling?

A series of recent articles by Steve Denning on Forbes have highlighted the challenges that the Agile community faces to get acceptance by mainstream management.

In the first article in the series, Steve makes the suggestion that the best-kept management secret on the planet is Agile. He argues that there is a trend over many centuries that bold new ideas that challenge a way of thinking are often ignored if they come from an unexpected source:

...Just over a decade ago, a set of major management breakthroughs occurred. These breakthroughs enabled software development teams to systematically achieve both disciplined execution and continuous innovation, something that was impossible to accomplish with traditional management methods... Unfortunately, these management discoveries were not made by “the right people”: academics in business schools or high-paid managers in big corporations. The discoveries were made by the people that, in prospect, you would think are the least likely people to have solved a management problem: geeks.

He goes onto observe that Agile references in the management field are scarce or obscure:

...The management world remains generally in denial about the discoveries of Agile. You can scan the pages of Harvard Business Review and find scarcely even an oblique reference to the solution that Agile offers to one of the fundamental management problems of our times.

Commenting on the article, Kurt Häusler raised the point that perhaps it would be better to introduce these ideas via approaches that make more sense to non-software managers. Steve replied with the following suggestion:

...Ignoring the management experience of thousands of software development teams around the world is what doesn’t make sense. A better idea is a better idea, even if it comes from the “wrong” people. The approaches of beyond budgeting, complexity thinking, lean management and rightshifting are obviously related to Agile software development, but it’s not obvious that they are having any more real success in changing the basic command-and-control paradigm of traditional management.

Yves Hanoulle responded in the comments with an alternate view:

It makes me think of Jerry Weinberg’s “Law of Raspberry Jam”. The wider you spread an idea the thinner it gets. And that also happened to waterfall. Most people implementing waterfall have never read the Royce document (if they had they would be doing more iterations). In my experience, CFO’s and upper management are not the ones that feel threatened about Agile. When you have the chance to explain to them, they usually understand (and agree) very fast. That is why we need publications in management literature.

In his second article, Steve Denning interviews Rod Collins to try and answer the question of why can't the C-Suite grasp Agile management. Rod admitted that Agile was a surprise to him and he made the connection that there were three principles that were common between the new management paradigm and Agile Management:

• A riveted focus on what the customers value and what’s most important to them.
• A preoccupation with group process rather than distributed tasks as in traditional management.
• A recognition that work is fundamentally iterative learning

Rod continues to explain why C-Suite leaders just don't get approaches like Agile:

...When you explain this to senior leaders in organisations, they have difficulty “getting” it. Their experience of power is almost exclusively around the notion of being in charge. In the new management model, power shifts from the notion of “being in charge” to the notion of “being connected”. This means that you don’t have to be in charge to exercise power. What’s interesting about the new organizations is that they tend to be more powerful than the old organizations.

Chris R. Chapman responded to this second post in the comments stating that this is nothing new:

The problems of perception and inability to understand the agile mindset have stymied true transformation of the software industry for since the first agile projects began over 20 years ago. Management sometimes needs a mirror held up to their behaviours to see the impracticality and how current practices are contributing to the misalignment between our work and how we do it... The shift in mindset needs to occur organisation-wide to realign workers and managers with their work. This means becoming entrepreneurial and embracing risk and change – even within established organisations.

In response to the discussions, Steve Denning wrapped up the series with a look at "The Case Against Agile: Ten Perennial Management Objections" (six of the ten objections are based on a 2009 article called "Limitations of Agile Development" written by Bruno Collet). The ten objections, and a summary of his key thoughts are as follows:

1. “Agile is only for stars”
Faced with the choice between high-performance and the mediocrity, traditional management opts for mediocrity.

2. Agile doesn’t fit our organizational culture”
In today’s marketplace, they will need to change their culture or they will die. They need to become Agile.

3. “Agile only works for small projects and our projects are big”
There are obvious solutions to coping with large projects by dividing the work into a number of relatively independent smaller subprojects then each part can be implemented by an agile team.

4. “Agile requires co-location and our staff are geographically dispersed”
While it’s true that co-location is obviously ideal for Agile teams, where staff are dispersed, teams can use technology maintain open and continuous communication.

5. “Agile lacks project management processes”
Unless you choose an agile methodology that encompasses all needed processes, you should combine it with a methodology that define these processes and rely on agile for day-to-day team management.

6. “Our firm’s individual accountability systems don’t fit Agile”
Change the organization’s reward system. They are the problem, not Agile.

7. "Agile is just a fad”
It’s not a silver bullet... Agile is the solution to a particular problem, namely, reconciling disciplined execution with creativity and innovation.

8. “There are better ideas than Agile”
Rather than fighting internecine battles... it would be more useful to see the commonalities in these various movements and join forces to get real change.

9. “Nothing new here”
All of the individual components of Agile have been around for quite a long time. What is new is to put those elements together in a coherent and integrated fashion.

10. “Not a fair comparison?”
Introducing (real) Agile means exposing all of the non-transparent tricks that hierarchical managers play on their subordinates to maintain power.

Yves Stalgies eloquently wrapped up the conversation in his comment:

...Agile is a mindset, not a method and that’s the important point. And what is needed is a change in mindsets. The Agile mindset is very common in software development now management has to change its mindset. The Agile mindset and the customer delight mindset are the keys to fulfilling and successful companies.

Steve Denning in his book "The Leader's Guide to Radical Management" explores in more detail the ways in which traditional management needs to change (InfoQ has an in-depth review of the book by Shane Hastie).

Is Agile adoption being stifled by traditional management because they are unaware or more because they are unwilling? What actions can we take as a community to change these perceptions?