Opinions: Measuring Programmers' Productivity
- Share
-
- |
Read later
My Reading List
In the field of software development, just as in any other area, managers need to appreciate performance of their programmers and the progress of their projects. However, defining appropriate metrics to this end appears to be a tricky task.
Measuring source lines of code (SLOC) is one of commonly used approaches that presents however a number of important limitations highlighted recently by Shahar Yair and Steve McConnell. First of all, the amount of lines of code does not allow effectively measuring the progress of a project because it focuses on activity rather than results. LOC do not have any value as such: the value of the final product depends on its performance and quality, not on the amount of code. Hence, focusing on the latter is actually a very limited approach to productivity.
SLOC doesn’t tell anything about the complexity of the issue at hand or about the quality of the final product in terms of maintainability, flexibility, extensibility, etc. With regard to quality, it can actually be negatively correlated. Refactoring as well as some design patterns result in reducing LOC while improving code’s quality. Larger code base may mean more noise, more unnecessary complexity, less readability and flexibility.
What is particularly risky about adopting such a reductive view of programmers’ performance is that it creates wrong incentives. Rather than optimizing their work in terms of final product, developers may be encouraged to favor the quantity of code to the detriment of quality and even intentionally write more verbose code. “What gets measured gets done”, recalls Steve McConnell.
He points out that some of these issues can be solved by using function points as measuring metric. The program size is then determined by the number of inputs, outputs, queries and files. Nevertheless, this approach also has its downsides. McConnell mentions some practical issues like the necessity of having a certified function point counter and the difficulties with mapping each function point back to individual programmers. A certified function point specialist Daniel Yokomizo highlights in his comment other limitations of such approach: lack of tools to measure function points’ complexity and to take into consideration things such as code sharing, frameworks, libraries, etc., that affect the time needed to create a feature.
Even though many commentators involved in the discussion about measuring approaches agree on their limitations, they do not necessarily dismiss the need for measuring programmers’ performance. Many insist on the fact that SLOC, for instance, can be used as a baseline for a more complex analysis combining different factors. This goes along the lines of four principles outlined by McConnell that should guide any analysis of programmers’ productivity:
1. Don’t expect any single dimensional measure of productivity to give you a very good picture of individual productivity […]
2. Don’t expect any measures […] to support fine-grain discrimination in productivity among individuals. [They] give you questions to ask but they don’t give you the answers […]
3. Remember that trends are usually more important than single-point measures […]
4. Ask why you need to measure individual productivity at all […]
In which context measuring programmers’ productivity is actually meaningful? What criteria can be used for it? How can they be combined? Many questions are still open for discussion and if your experience has brought you some answers, don’t hesitate to share.
Related Sponsor
Agile Project Management That Adapts With You. Provide your global development team a shared space that adapts to the way they work.
A minor nitpick
by
Daniel Yokomizo
Why do I care?
by
Bruce Rennie
Where does the desire to measure developer productivity come from? Do they measure the productivity of doctors? Engineers? Plumbers? And those don't tend to work in teams.
LOC
by
Stefan Wagner
If you bundle some subprime measurements together, do you get a triple-A measurement, or do you get junk?
Maybe I would start writing code-generators, if LOCs would be measured, but most probably I would leave.
Re: Why do I care?
by
Sadek Drobi
Re: Why do I care?
by
Thomas Cagley
Tom Cagley
www.spamcast.net
Re: Why do I care?
by
Bruce Rennie
Maybe doctors do have productivity measurements (patients seen per hour), but they also have a strong professional code that trumps any productivity measurement. Doctors talk about lives saved, not surgeries per day. Ditto for engineers: only the bridges that are still standing count.
As I also pointed out, things get even more fuzzy when a team is involved. In a team environment, measuring and optimizing at the individual level feels like I'm ignoring Theory of Constraints. I should be measuring and optimizing for overall throughput.
As a final question: does measuring productivity on a failed project make any sense?
Re: Why do I care?
by
John Reynolds
- actual development time compared to estimated time
- re-work / bug fixing time vs. actual development time (this can be useful if you think about it)
- customer satisfaction / feedback survey
John
Danger - will destroy teams
by
Mark Levison
1. Spend time in planning/retrospective meetings
2. Mentor jr team members
3. Pair Programming
4. Code Reviews
5. Simplify code, remove duplication, anything that might reduce my function point count
6. Anything else that would take time away from creating function points
Further more I will learn what function points are and learn to game them by writing code that maximizes the number of function points.
Find a measure of the individual and I will show you how it destroy's teams. Net result I recommend against measuring individuals or even teams. Instead I focus on outcomes.
Hudson Continuous Integration Game
by
Zachary Shaw
The basic gist is that you get / loose points based on your checkin's. The more you checkin, the more tests you write, the more warnings / TODO's you clean up, the more points you get. It's true so far here "What get's measured get's done", warnings have gotten cleaned up, more tests have gotten written, and people checkin more frequently. It doesn't necessarily measure who's really productive, but it measures who is creating future work, so maybe in a way that is a measurement of productivity.
Re: Danger - will destroy teams
by
Zachary Shaw
I think it's important to measure things that you care about your team doing. I also think it's important for the team to agree on what they're being measured on. If you get team buy-in then it can lead to healthy competition.
I do not think that SLOC is a good measurement. It doesn't speak to quality. But there are a bunch of static analysis tools that do speak towards quality, that hook into a build. I think the idea of the Hudson game is to use that information to measure developers. Honestly, we've just started using the Hudson game, and it seems good, so I may be a little blinded by a new shiny toy, but can anyone see a downside to the Hudson game?
Re: Danger - will destroy teams
by
Mark Levison
I'm familiar with Hudson and think it an excellent way of highlighting quality issues as long as the information is kept private and within the team. I would be concerned if the team started to take it too seriously or management got involved.
Re: Why do I care?
by
John Jimenez
Thanks.
Re: Danger - will destroy teams
by
John Jimenez
Ideally one would prefer a culture to perform come from peers directly, through healthy competition, peer evaluations, etc. However, if this doesn't exist from within then it's management's job to make is so and productivity metrics are a vital tool.
Re: Danger - will destroy teams
by
Mark Levison
What really matters here is the output of the team and are the team happy working together. If the team is producing high quality well tested, simple code - why does it matter how productive they are? If they're happy then who cares that one team member is really just acting as the glue.
When I coach I encourage management to set goals for the team and measure whether they meet those goals. Anything done on the individual level will destroy the hard won gains at the team level.
Know why you're measuring, and keep it squeaky clean
by
Robert Merrill
I ran the metrics program for a small (15-20 developers) project shop for 5+ years. We measured productivity for only one reason--project estimation for client proposals. We estimated two ways, using an agile-style planning game, and also using function point analysis. (I was a CFPS from 2002-2005). I counted each project as built, and used the time records to determine h/FP.
When it was time to estimate, I would count the requirements and consult the project history to predict h/FP. Having an independent estimation method was very helpful, and it also held up better under client pressure than the fully subjective methods. (Function Point Analysis is also an excellent way to spot requirements gaps).
To keep the whole thing squeaky clean (even though it was a high-trust culture already), I was the only one with access to the project history. Management never saw it, except in summary form.
Metrics are an amplifier. They will make a high-trust organization even more effective, and will make a low-trust organization even more toxic.
Quantitative performance measure will be the disaster to team. Be careful!
by
Wu James
Deming said that you never measure your people in digits. That is not only true in manufactory, but also in SW industry. SW development is more complicated than assembly line.
When you link personal performance with individual's productivity, you definitely can get the number increased. However, it is not your revenue or quality. Your people is smarter than you. They can figure out everything to make your happy to increase their salary and bonus.
As Bruce said, "Why do I care about programmer productivity?". Exactly right. We are selling quality products not the code lines or even funtional point. Sofware is not built by blocks.
If you DO want to measure sth of performance, following DON'T/DOs maybe useful:
1. DON'T use the digits that the team can control. Use inputs or outputs from/to customer. Do link it with your revenue.
2. DON'T compare data across teams. Do compare it with your past to see the trend.
3. DON'T use complicate/fancy methods. That will definitely will increase overhead, or effort waste, to your team. DO use some indicators which are really simple and easy to collect.
If you can use your leadership, forget the digits.
Educational Content
Writing Usable APIs in Practice
Giovanni Asproni May 19, 2013
Hello stranger!
You need to Register an InfoQ account or Login to post comments. But there's so much more behind being registered.Get the most out of the InfoQ experience.
Tell us what you think