Is the Agile Community Being Unreasonable?
The pmiagile Yahoo! group is a place where the two communities come together. The contributors to this group come from a wide variety of backgrounds. A recent thread discussed the applicability of agile practices in traditional project management environments as described by the project management body of knowledge (PMBOK). (This reporter apologizes in advance for the long quotes, this is an intense topic that was difficult to paraphrase.)
About once per season I encounter the belief that the PMBOK doesn't contradict agile approaches. It would be nice if that were true. (reposting) Here are some quotes from the PMBOK Guide Fourth Edition suggesting the unsuitability of "project management" to product development with Scrum:
- "Projects are not ongoing efforts."
- "The purpose of a project is to attain its objective and then terminate."
- " Recognition and rewards. Clear criteria for rewards and a planned system for their use will promote and reinforce desired behaviors."
- "Part of the team development process involves recognizing and rewarding desirable behavior. The original plans concerning ways to reward people are developed during Human Resource Planning (Section 9.1). Award decisions are made, formally or informally, during the process of managing the project team through performance appraisals" "For example, if the schedule is shortened, often the budget needs to be increased to add additional resources to complete the same amount of work in less time."
And Wayne Mack shows his frustration with the Agile community as he understand their recommendations of letting the teams self-manage:
It seems a bit brash if our recommendations to Project Managers is to not manage projects.
My current work environment is very accurately described in Michael's post below. Before I can start a project or even have team members assigned, I need to present a schedule, budget, and scope for approval. Even with customer requested and approved changes to scope, I need to justify changes in schedule and budget. This is why many project managers find suggestions of dynamic scope as almost incomprehensible.
To vent a little, I find APLN and the Agile COP interesting and reinvigorating, but I am not finding information that is applicable to my day-to-day work activities. I often feel like I am in a one man struggle and I would like to share information with others in the same situation.
It is my hope that we can discuss various strategies for applying Agile to PMI PMBoK project-based work. This would provide true benefit to Agile COP members and to PMI members at large.
Brad Murphy acknowledges Wayne's comments and elaborates on the Agile communities weak point:
Please know that you are not alone. Having spent 10+ years advocating and helping large, mainstream companies initiate and sustain a transition to "agility" I echo your sentiments regarding the Agile "purists". While I am a HUGE fan of many Agile principles and engineering techniques, the APLN and other "Agile" professional groups are often naive and sometimes down right dangerous in their disregard for the complex realities of organizational change management, governance, financial controls and longer horizon planning activities.
Here's a simple example of the naivete I'm referring. A certain Scrum industry leader says that self-empowered teams have a "right" to expel a team member for under performing. This industry leader advocates that In the event of a poorly performing contributor the team should counsel the team member together and if the team doesn't feel the colleague is correcting his/her behavior then the team is empowered to expel this person from the team. Really? In any larger organization this will result in legal wrangling, lawsuits and HR raining down like none other.
I'm not suggesting self-empowered teams shouldn't have a voice in shaping their membership, but this simple minded solution is not respectful of the existing legal and organizational constructs that must be reshaped in a more constructive manner than by being dismissive of current practices.
You tell a story about dysfunction. Literally. On the one hand, you have Scrum encouraging a team culture of productivity. On the other hand..... This inability to expel unproductive people (which confirms a productive culture) is "baked-in" corporate dysfunction:
This is literally THE problem. If a company cannot confirm it's own self-selected (new) culture of productivity, for example by repelling and removing lazy people, what then? The answer is, THAT firm continues to attract and hire new lazy people, while confirming that existing lazy people have a right to be present.
This brand of agility is optimized for very long consulting engagements.
If on the other hand the firm can express the (new) culture by removing lazy people, then that firm takes flight, embodies productivity, and no longer needs an army of embedded "agile" consultants who cultivate dependency. This is because the firm "gets it" and no longer needs the consultant.
How many consulting firms that provide "agile coaching" actually serve customers by encouraging client self-awareness and thus independence... from Day1 ?
Very few. Maybe none.
Thanks for this intense discussion. It was so interesting and engaging that I feel compelled to engage. :)
I can empathize with your frustration, having faced such challenges and myopia in certain circles. For example, there’s been an ongoing dialog about the role of management on self-organizing teams. See my Blog post http://lithespeed.blogspot.com/2009/11/self-organization-self-discipline-light.html for my own perspective. Wayne — I would echo Brad’s sentiment that you are not alone. There are several of us who have spent the last decade helping bring agility to large, mainstream companies. I would think that is why (at least in part) large, mainstream companies are going agile in an increasing way.
I also think it’s good to vent, so that we can hear what folks are really thinking out there and engage each other in productive discussion. However, I would caution against sweeping negative generalizations of any community: APLN, Scrum or the PMI. As someone engaged fairly closely with all three (Co-founder and current VP of the APLN, active CST, and Member, PMI CoP), I think folks in all the named organizations are making sincere and effective moves to take agile mainstream. I also believe that they are all sincerely working towards a pragmatic view of agility, albeit from their own frames of reference. Here are some anecdotes that should support my contention:
- The CEO of the PMI was the keynote speaker at last year’s U.S Scrum Gathering. Close to half the attendees at that Scrum Gathering (as judged by a show of hands requested by Mr. Balestrero) were PMPs.
- Personally, I have never seen or experienced the naïveté allude to at the APLN — either on the Board or at any of the chapters at which I regularly present. My present and past colleagues at the APLN: Jim Highsmith, Pollyanna Pixton, Bob Wysocki, Susan Fotajek, Kent McDonald and Todd Little to name a few, are all grappling with the same challenges you raise and take a very similar enterprise view as I do.
- I have presented the same Agile PMO session that addresses governance and program management at the APLN (see http://www.aplnhouston.org/), the PMI (see http://www.pmiwdc.org/careerday2009-education#Sanjiv_Augustine) and the Scrum Gathering ( http://www.scrumalliance.org/events/105-orlando-scrum-gathering). Interestingly, I received very similar positive feedback from all 3 audiences.
While there are some voices in the agile community whose opinions might not hold sway in today’s corporate circles, I would respectfully request that we be careful not to paint everyone with the same brush. At the same time, I would also point out that these views are what move us to improve and get better as a whole even if we find them controversial today. For instance, we might question a Scrum industry leader’s belief that a self-empowered team should be allowed to select its own membership. Yet, in fact, that is exactly the norm at Whole Foods Corporation (From http://money.cnn.com/2007/09/26/news/companies/management_hamel.fortune/index.htm, “the underlying logic is powerful if unconventional: Whole Foods believes that critical decisions, such as whom to hire, should be made by those who will be most directly impacted by the consequences of those decisions”).
So, to some, the Agile community is ideal or naive. Perhaps they have a point. Perhaps Agile is not well suited for large environments. Or, just maybe, Agile techniques are really good at creating confrontation and bringing things to the surface that have been there all the time, but have not been painful enough to confront. Thoughts?
(1) Conventional assumptions and practices for IT management, governance, and so forth have always been flawed. They are the cause of nearly every problem companies have with IT. We have become so accustomed to a 16%-29% success rate (per the Chaos reports of 1994 and 2003 respectively) that we've merely redefined that level of performance as "success," so that we won't cry ourselves to sleep every night.
(2) "Agile" came from the trenches of software development. It's specifically designed for software development activities. It says so right there in the good old Manifesto, plain as day. "Agile" doesn't offer mechanisms to solve organizational or management problems. The fact agile works so very well for software development doesn't mean it works just as well for other things.
The two sides of this debate are only defending the point of view that each prefers. Neither appears to be looking for real answers to the real problems in IT.
Michael James quotes a few lines from the PMBOK. I don't see any conflict between these quotes and agile values or principles.
PMBOK says projects are not ongoing efforts. There's no error in that statement. It's quite right. The problem is that IT organizations plan their work as discrete projects. PMBOK treats this as a "norm" instead of as a problem. That's because the P in PMBOK stands for "project." If you want to manage the work in your company as a set of ongoing product development work streams or as value streams, go ahead and do so. Just do it without PMBOK. No conflict.
Recognition and rewards. PMBOK doesn't say how, it only says what. We can apply team-oriented rewards for team-oriented work styles like agile, and individual rewards for individually-oriented work styles. No conflict.
PMBOK allows for increasing the budget when the situation calls for it. As I read it, that means PMBOK doesn't lock us into an annual budget cycle (another problem posing as a "norm"). We could use a just-in-time budgeting method and still remain consistent with PMBOK. No conflict.
Wayne Mack says: "Before I can start a project or even have team members assigned, I need to present a schedule, budget, and scope for approval. Even with customer requested and approved changes to scope, I need to justify changes in schedule and budget."
So, we have: Annual budget cycle. Preapprovals far in advance. Fixed scope. Fixed schedule. Team members assigned on a per project basis (part-time basis, too?), teams disbanded after each project. Concept of an internal "customer," with the administrative separation the terminology implies.
Holy cow! Never mind "agile" or any other popular buzzwords: How many things can a company do wrong before it goes out of business?
"I am not finding information that is applicable to my day-to-day work activities."
Yeah? Go figure.
Brad Murphy says, "[An] industry leader advocates that In the event of a poorly performing contributor the team should counsel the team member together and if the team doesn't feel the colleague is correcting his/her behavior then the team is empowered to expel this person from the team. Really? In any larger organization this will result in legal wrangling, lawsuits and HR raining down like none other."
I can imagine how a person unfamiliar with agile team dynamics might make this sort of assumption. Experience makes certain things a bit more clear:
First, there's no question of a team "firing" anyone. Self-organizing teams don't have that level of authority. Technical people don't even want it. It isn't what they do.
Second, there has to be management support for team self-organization. Teams can't just wake up one fine morning and "be" self-organizing. The manager has to be savvy enough to allow the right degree of autonomy to each team, to let them learn from mistakes, to coach them, and to set clear boundaries; there's no PMBOK-style cookbook for this style of management.
Third, the process of counseling the team member is typically supported by the manager establishing the groundwork with HR, in case the counseling doesn't work out. HR is normally fully on board from the start. I've seen this sort of situation on a number of different teams. It hasn't been exactly the same in any two cases.
In the best example I've seen, a team had one member who didn't seem to be picking up on any technical skills or domain knowledge. After a few weeks, the team dedicated a retrospective to the matter. They all agreed (including the low performer) to a one-month program of mentoring, after which they would see how things were shaping up. They adjusted their iteration commitments downward so that they would be able to dedicate some time to helping their teammate. The manager agreed to support their plan. She talked to HR and let them know what was going on. The team's Product Owner understood the plan and accepted the lower delivery commitment for the one-month period.
The guy did his best to get his skills up to snuff. Ultimately, he wasn't able to rise to the challenge. He found another job in a less challenging environment where he did well. It sounds a bit trite, but it's actually quite true that everyone needs to find an environment where they can thrive. An agile team in a fast-paced, technically-challenging environment isn't the best place for everyone. One person might find it energizing and another might find it overwhelming. It's also true that no one in the organization is in a better position to see such a problem than the person's teammates; his pairing partners. "Expel" is probably too strong a word for this process, but the basic premise is sound.
This actually works better in a larger organization than a smaller one.
When an individual doesn't fit well in a team, a larger organization can offer more alternatives to him/her than a smaller organization can do.
Well, long story short: If "agile" doesn't solve all the world's problems, that in itself doesn't mean we need to scurry back to the 1980s as fast as our little legs will carry us.
I haven't laughed as much in ages. I thought I was going to split my sides at one point :) Thanks for brightening up my day.
One point though, I'd be careful in using the Standish Chaos report figures, and using this as an argument for driving through change. Although still sometimes referenced in literature, the 1994 report has largely been discredited as bad science, and more recently their method criticized for presenting an overly biased view of software project failure rates, by Robert Glass, Magne Jørgensen and others.
I'm aware of the criticisms of Standish's methodology, and your point is well taken. However, their results do seem to line up pretty well with my own experiences and observations in IT; and I don't think you'd have much trouble finding a lot of other people who have similar observations - especially those who have been "customers" of IT departments. I remember a comment by the Product Owner of one of the first agile projects I worked on: "I've been working with IT departments for 32 years, and this is the first methodology I've seen that actually works."
I wouldn't want to get bogged down trying to perfect a statistical analysis method to the extent that I forgot to look at what was actually happening around me! If someone is poking you in the eye with a stick, do you really need to wait for a methodologically-sound "study" to be published to prove that someone is poking you in the eye with a stick?
A Shaw quote comes to mind: "Reasonable people adapt themselves to the world. Unreasonable people attempt to adapt the world to themselves. All progress, therefore, depends on unreasonable people."
Great follow up comments, well made. A very pragmatic look at what agile might offer over older styles, and equally on what it never said to offer within large organizations.
"It seems a bit brash if our recommendations to Project Managers is to not manage projects."
That sounds confusing but that's often the best advice you can give Project Managers. If it's not a Project but an ongoing effort, don't treat it as a project.
BTW: How is PMBOK pronounced? Humbug?
The PMI (and the PMBOK) are meant to be a toolkit that recommends that as a project manager you ask the right questions and focus on the right areas to ensure project success. It should not be viewed as an exact blueprint that has to be followed. The tools I would use for managing a construction project are different than what I would use for software development, but the general framework can ensure that we do the right thing in either case. Risks, stakeholders, change, scope, etc. all need to be managed in a corporate environment, but the rigor and process/methodology should adapt to the project needs. Factors like Regulatory compliance and government contractual requirements often add requirements and sometimes constraints to development teams
What Agile often exposes is inefficient or overly rigid corporate processes, often enforced by a PMO, that often were built around reporting rather than executing needs. The process and reporting become self-serving and extremely difficult to change because companies will often have invested significant time and money to create and protect them. I have found it very effective to educate senior leadership in agile and allow them to experience the culture change required to support the bold steps to eliminate these historical roadblocks.
While I believe that Agile and PMI do fit together, I know there are many on both sides that view them as polar opposites. My stance is that either, if taken strictly by the book and applied without thought or experience, will experience some failures when scaled. The key is to inspect and adapt. Failure is only a failure if you fail to learn from it and allow the failing behavior to persist.
I encourage our Project Managers and ScrumMasters to take the approach of being "gap fillers" and servant-leaders in that they work with the team and find the areas where the team needs the greatest support and fill those gaps. The methodologies they use, both from an Agile and a PMI perspective, should adapt to the business need. Just as whether the team determines it is best to use Scrum or Kanban, the project managers are expected to understand the level of rigor of change control, risk management, etc. required to meet the client's needs. In all cases, doing just enough process where they need to land.
What issue are we confronting here
1) For those of us to claim to be part of the agile community, we are not in agreement. Name calling aside, this divergence is good for finding various solutions, but is also confusing to those trying to figure out what to do.
2) There was a question/answer pair given by Wayne above: a) Why does Agile want the ability to expel members off a team? b) This cannot possibly work and is naive. Are there large organizations that have tried this and failed? What were the results? Are there large organizations that have made this a rule? Sanjiv points to Whole foods, I've read about Semler's work in Brazil, are there others?
3) There is a clash of ideals and cultures between mainstream PMI and mainstream Agile. It sure would be great if we could be precise about them, then perhaps we can address them one by one.
Finally, anyone who has something to say about these topics and would like to write an article for us at infoQ, we'd be happy to publish it!
There are also massive problems in bringing scrum into any organisation - and these problems are all in the middle management - the top managers can see the massive lack of productivity in an organisation, the engineers are frustrated by the paper shuffling stifling atmosphere, but the middle managers don't want the change as they are scared about their jobs.
PMBOK and similar are about promoting paperwork and control, its all about 'planning' and 'controlling' not about 'doing'. Scrum is about doing, its about getting the people who are going to create the solution the time, space and enthusiasm for doing so.
Updates to a product are often looked at as individual projects, and I don't have a problem with that, scrum produces shippable product regularly, any one of those can be chosen as the end point of a project and let loose on the real world. Self organising teams, a direct link between the team and the person who is specifying (selling) the product, good communications are the cornerstones of the productivity scrum provides, these are the reasons scrum doesn't need a whole raft of middle managers, and why they are so scared.
Middle managers aren't dumb, they have a cushy, comfy position, well paid, a large empire, plenty of paper to shuffle and meetings to attend so look really busy, someone to blame if it doesn't work, the ability to grab the glory (bonus and payrise) when it does work, and above all no stress at all.
Re: What issue are we confronting here
I think one issue that is exposed by this discussion is at a level of abstraction higher than the specifics of PMBOK or agile or Scrum or what-have-you. It's the difference between choosing a particular named, branded package of ideas and defending it religiously versus looking at each organization's goals and needs and then determining a practical path for improvement.
Each of the approaches / frameworks / toolkits that people mention during discussions like this has its own strengths and weaknesses, and is designed to solve certain problems at a certain scope. When the debate is about whether "agile" works in large organizations or whether PMBOK is about "control", we miss the opportunity to identify real problems and discover practical solutions. We end up in a pointless, circular, quasi-religious debate. In that case, there's no reason not to be tongue-in-cheek or sarcastic about the debate, because it has no chance of resulting in anything useful anyway. It might as well serve as entertainment, if nothing else.
We're all fond of declaring that there's no such thing as a magic bullet, and then many of us revert to defending our favorite magic bullet. Real solutions may call for elements of agile, elements of PMBOK, elements of lean, elements of Six Sigma, elements of CMMI, elements of Systems Thinking, and many other disciplines or schools of thought, with each applied at the appropriate scope and to address the problems it was designed to solve.
"Agile" works in any size organization, not because it "scales" but because it supports software development activities regardless of whether the organization has one team or a thousand teams. "Agile" supports software development very well. It doesn't do anything else. That's not a flaw, because it's not supposed to do anything else. When I hear people say "agile" has failed because it didn't help them with portfolio management or strategic planning or choosing a tie to match their socks, I can only shake my head in bemusement. It's as if I bought a new electric nail gun and then complained about it because I couldn't use it to bake a cake. What a crappy nail gun. It won't even bake a cake!
By the same token, as others have pointed out, when managers use PMBOK as a cookbook or checklist in a rigid way, they end up pursuing the illusion of control with no view to the organization's goals or problems. When managers use PMBOK as a general framework to help them keep sight of important management functions while remaining flexible about management style, project planning and reporting, and other implementation details, then they can support a heterogeneous organization that uses different processes for different work flows.
There's no conflict except between quasi-religious adherents of a given approach or framework. This is not a genuine clash of ideals or cultures. It's just stubbornness and limited vision.
Re: What issue are we confronting here
There are a lot of differences, but the only *conflict* I see between PMBOK and agile methodologies, is basically one of underlying philosophy. In PMBOK, the Project Manager is the project's hero and the Plan is sound (reality is wrong - when it doesn't conform, corrective action is taken). In agile, the project manager is a team member like any other, and reality always trumps the plan (so when they don't match, the plans are revised). This is a bit over-simplified, but it's just to illustrate the conflict.
However, it's not PMBOK or a methodology that decides the philosophy and the tone of the project - it's the people in and around the project.
Personally, I've learnt more useful stuff from the agile methodologies (for instance multi-level planning and useful estimating techniques) but I'd never want to run a project without the more disciplined risk management approach in PMBOK. But that's me, we each have our own path.
Is this true ??
There are a lot of differences, but the only *conflict* I see between PMBOK and agile methodologies, is basically one of underlying philosophy. In PMBOK, the Project Manager is the project's hero and the Plan is sound (reality is wrong - when it doesn't conform, corrective action is taken). In agile, the project manager is a team member like any other, and reality always trumps the plan (so when they don't match, the plans are revised).
This can't be true - especially the part about reality. What say all the PMI folks out there?
One of my favorite questions from a PMI course I took was, "If the PMBOK is 'The Guide to the Project Management Body of Knowledge', where is the Body of Knowledge?"
The answer was/continues to be... "You are."
In that simple question and answer is the summary of what the Guide to the PMBOK is about: it is a merely a guide, a compendium of best practices (tools and techniques) for you to use in your daily struggle to get things done.
A related quote that I share from Alistair Cockburn from a thread where Agilists and PMPs were debating the approach to system testing on a software project: should it be a separate group or should it be part of the Agile team? Cockburn replied with the one liner: "Do what makes sense."
Isn't that what both the Guide to the PMBOK, the Agile principles, and your heart tells you to do? In this, there is harmony between the PMBOK and the Agile principles. In this, there is also harmony between the unholy triumvirate of the PMBOK, the Agile principles, and CMMI.
Maybe it's just me; however, I find myself often stopping meetings in mid-debate to ask the simple question: what makes sense? It cuts through team dysfunction, sacred cows, and brings the needle on my baloney meter back down into the tolerable range. Feel free to pick up your own Carl Sagan inspired "Baloney Detection Kit" here.
Re: Is this true ??
1) PMI does champion the project manager as a leader who should use their formal and informal influence and authority to find resolutions to project issues.
2) Corrective action is taken when a project is not conforming to plan. Sometimes this is to change the plan, while other times it is to help get back on plan. PMI does not say what to do, only that you need to determine the variance and then take appropriate action. I don't see this in conflict with agile as it really boils down to resolving blockers that arise.
The biggest difference between Agile and the PMBOK is the amount of recommended planning up-front. PMI does encourage fixing Scope, Time, Cost and other constraints and "control" them within acceptable tolerance ranges. Agile is obviously looser in the control area, particularly in Scope (embracing change).
PMI does discuss "Rolling Wave Planning" as a means to avoid heavy big up front planning, but rather execute in phases. PRINCE2 (UK PM Methodology) has Stages and Stage Gates that roughly amount to the same thing. The key with any PM Methodology is to use the appropriate level of planning for the project requirements. Most of my planning for larger Agile projects is typically around creating ground rules and communication strategies for dealing with large project issues, not team software development where Agile is proving to be significantly more effective than traditional waterfall.
Re: What issue are we confronting here
I agree 100% with your point. It all boils down to taking a pragmatic approach for the project and environment you are working in.
Software Engineering is a vast domain ranging from safety-critical systems to simple HTML web-sites where these different systems can be implemented by teams from different cultures, organisational sizes, and skill levels.
The concepts presented in Barry Boehm's Balancing Agility and Scale can be used as a basis (bit.ly/8Xqfuf) to determine the best suited approach for a given environment.
To build on that, I believe, one should not discount the value of learning gained in Agile approaches such as taking an empirical approach and running self-organising teams.
Software development is not simple and one would be foolish to align yourself to one and only one approach to software development.
Re: Is this true ??
In agile, the project manager is a team member like any other, and reality always trumps the plan (so when they don't match, the plans are revised).Which is all fine, if you're in an environment where you don't have a constraint of fixed plans. Some projects really do have (and indeed *should* have to be able to meet the business outcomes) commitments for time, budget & scope that get locked before product construction starts.
In PMBOK, the Project Manager is the project's hero and the Plan is sound (reality is wrong - when it doesn't conform, corrective action is taken).The interesting thing is that this doesn't per se exclude adjusting plans, if the constraints permit.
Forgive me, but I'm seeing a false conflict.
Eran Kinsbruner May 05, 2015