Is Measuring Hyper-Productivity a Waste of Time?

Posted by Vikas Hazrati on Jun 02, 2009

Sections

Process & Practices

Topics

Agile in the Enterprise ,

Agile ,

Agile Techniques ,

Teamwork

Tags

Best Practices

In a presentation about Hyperproductivity and Shock Therapy, Jeff Sutherland mentioned that Hyper-Productivity is at least Toyota level of performance which is four times the industry average. According to him, teams doing Agile development the right way, see a 300% improvement in 3 two-week sprints. In a recent discussion on the Scrum Development group, members debated whether it is both fruitful and possible to accurately measure productivity across sprints. Also, whether it is possible to identify if a team has reached hyper-productive state.

Adam Sroka suggested that it is difficult, if not impossible to measure productivity across sprints. Most of the times teams base the productivity gains on story points, which does not remain constant across sprints. According to him,

Story points are not necessarily comparable, although they seem to converge over time (e.g.five story points in iteration one is likely not comparable to five story points in iteration five. However, five story points in iteration five likely is comparable to five story points in iteration six.) So, the increases in productivity we see through the course of an Agile project are anecdotal. We have seen them. We know they exist, but we lack any definitive way to measure them.

Tobias Mayer agreed to Adam when he mentioned that,

This "hyper-productivity" measuring is mostly a waste of time -- and a way of selling snake oil. Estimation of stories is not a constant thing; teams change the way they do this over time as much as they improve their ability to deliver. And as Adam points out comparably complex stories require less and less effort as we get better.
I think such "quantified" claims do a disservice to Scrum, and as the original poster points out, if you set your baseline low enough, everything can appear as hyper-productive. What if a team completed nothing at all in sprint one? Then getting one story point done in sprint two would represent an infinity% increase. It is all nonsense.

Joseph Little suggested that though a lot of talk about hyper-productivity is gush, hyper-productivity in itself does make sense. He suggested that in order make improvement the team should know their velocity. He agreed that the velocity might change with the iterations and so would assigning different story points to a similar sized story across sprints. The way to manage this to average out and consider the average velocity over 3 sprints rather than one. The next challenge should be to double this velocity by making minor and major changes in the development process so that the impediments are reduced considerably. According to Joseph,

Let me be a tad blunter: Things are so screwed up in SW dev, that doubling a team's productivity is not that hard.

Roy Morien added that it is very difficult to measure personal or team productivity mathematically. Hence, instead of trying to achieve the measurement part, teams should focus on delivering useful, working software and try to improve on what they did in the last sprint.

On similar lines, Adam Sroka mentioned that instead of wasting time in measuring productivity effectively and analyzing whether the teams have reached the hyper-productive state, teams should focus on the following

1. Are we delivering working software that is valuable – continuously, every iteration?
2. Are we expending effort in a manner which is sustainable? Can we continue to deliver at this pace indefinitely? Can we improve?
3. Are we wasting effort on things which do not contribute directly to the delivery of value? How do we eliminate such waste?

Thus, throughout the discussion, most members of the group suggested that there are more important considerations to Agile development than just comparing numbers across sprints to see if a state of hyper-productivity has been reached. Moreover, since there is no standard way of gathering numbers across sprints, comparing them becomes a futile excercise.

1. Back to top

   Hyper-productivity important?

   by Amr Elssamadisy

   
   Thus, throughout the discussion, most members of the group suggested that there are more important considerations to Agile development than reaching a state of hyper-productivity, which more often than not is based on some flawed estimation.
   

   
   
   If productivity is a measure of delivering value, and hyper-productivity means delivering more value faster, how can there be ANYTHING more important?

   Reply

2. Back to top

   Re: Hyper-productivity important?

   by Vikas Hazrati

   I think the gist did not bring about the exact meaning.
   
   Rather than concentrating on numbers to figure out whether the team has reached a state of hyper-productivity or not, which one may not be able to gauge/measure effectively, the team should focus on other important attributes like delivering working software faster, removing impediments, eliminating waste etc. All this would help teams be more productive than just comparing numbers across sprints to see how close or far they are from hyper-productive state.

   Reply

3. Back to top

   Re: Hyper-productivity important?

   by Vikas Hazrati

   Re-worded the conclusion to capture the essence of the discussion.

   Reply

4. Back to top

   This discussion misses the essence of Jeff Sutherland's work

   by Mark Levison

   My understanding is that Jeff took an industry standard measure of function points per developer per month (a number like 2). He then had his projects measured for function points per developer month. This is where his numbers come from. Its not as many in this discussion supposed a measure of story points - which isn't an objective measure.

   Reply

5. Back to top

   Re: This discussion misses the essence of Jeff Sutherland's work

   by Amr Elssamadisy

   It is always hard to read a powerpoint presentation, but it seems you have a point Mark. Function points, however, are something that we (the Agile community) rarely speak about (or understand?).

   Reply

6. Back to top

   Re: This discussion misses the essence of Jeff Sutherland's work

   by Vikas Hazrati

   True, I have hardly seen function points being used in any Agile project(Mark, do you have instances where they are used?). I believe that Jeff used function points to do an apple to apple comparison with the SirsiDynix numbers which were {probably} used when SirsiDynix did the project. We can check with Xebia if they used function points but if I remember correctly and on the basis of interaction of some of the team members who worked on this project, they used Story points.

   Reply

7. Back to top

   Re: This discussion misses the essence of Jeff Sutherland's work

   by Rahul Sah

   Well I had a good conversation about this with Jeff and what he meant by hyperproductive teams. I read one assumption which I dont agree to in the above comments - that the value of a user story point changes during a project substancially...well when teams do release plannings they come up with the points for the user stories...when teams start doing the implementation across sprints they self-orient and improve their performance and they see start managing more story points per sprint. A simple example - I can run 10 kms in 1 sprint, the next sprint if I can improve my performance I can do 11 Km in a sprint. When teams start changing the value of the Kilometer, the whole essense of Velocity is lost and it won't make any sense to even have a velocity cos it will be a random number. We have tried a simple idea which works - define a detialed user story point definiton and two user story point definition it always help us compare the story with the same benchmark, the more clear it is, the more effective it would be in measuring velocity.
   In terms of measuring hyperproductivity or productivity it makes sense to measure them only to know if as a team we managed to remove serious impediments and it really helped us. Not all teams can reach hyperproductivity.If you are already in a good structure at work, maybe the advantages are less in terms of productivity. If you measure the velocity in terms of user story points develiered by the team and accepted in a sprint by the PO then the changes in useful productivity are easily noticed.I have noticed them in projects running at iSense Prowareness(An agile development company)

   Reply

8. Back to top

   Re: This discussion misses the essence of Jeff Sutherland's work

   by Jack Harvey

   Wouldn't it be nice to live in a world where we don't have to measure or track anything? No worries about speed limits, spending limits, or how much food we should eat to maintain and healthy lifestyle. Maybe one day we can achieve such euphoria.
   
   Reality is measurements in Agile and Scrum shops are necessary and requested. When used appropriately they can add value to identifying areas to improve ourselves as well as identify issues we need to fix. In regards to the topic about hyper-productivity, I would like to lend a thought or two.
   
   The importance here is finding a way to see improvement over time. That can be ensuring you release better quality code (how do you know unless you find a way to measure), that you have more and more automated testing in place (again, you need to find a way to measure) and see how well a Scrum team is maturing both in Scrum practices, engineering excellence as well as productivity. If we get better as a team at removing impediments earlier, more effective development and testing practices, less interuptions, and so on we should see and want to celebrate growth in productivity (or through-put).
   
   At Release Planning as a team reviews and sizes all the stories, they might have the following breakdown:
   
   15 Stories sized as 1
   12 Stories sized as 3
   22 Stories sized as 5
   14 Stories sized as 8
   
   After Sprint 1 the team completes a total of 8 Story Points, Sprint 2 they complete 10, Sprint 3 11 and Sprint 4 12 story points.
   
   If the team is improving in all other areas in their process, they should be able to complete more and more story points in their Sprints, right?

   Reply

9. Back to top

   Re: This discussion misses the essence of Jeff Sutherland's work

   by Theron James

   What do you guys think of Scott Downey Roboscrum metrics for hyper productivity?
   - Velocity
   - Work Capacity
   - Focus Factor
   - Adopted Work
   - Found Work
   - Targeted Value Increase
   - Accuracy of Estimation
   - Accuracy of Commit
   
   I'm thinking of giving it a run.

   Reply

• Older >