Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Articles Measuring and Improving Software Development Productivity

Measuring and Improving Software Development Productivity

In the book Improving Software Development Productivity: Effective Leadership and Quantitative Methods in Software Management, Randall W. Jensen describes how you can measure and improve productivity in organizations. The book contains practices, models and case studies which help you to quantitatively support adoption of agile software development.

You can download a sample of the book Improving Software Development Productivity to get an impression of the book. The book has been published by Pearson/Addison-Wesley Professional, Sept. 2014, ISBN 9780133562675, Copyright © 2015 Pearson Education, Inc.

InfoQ interviewed Randall about measuring and improving productivity, the contribution of agile to productivity, benefits from pair programming and teams, knowledge retention in maintenance and commandments for communication.

InfoQ: What made you decide to write a book about measuring and improving productivity?

Randall: That is a tough question. My productivity measurement efforts began in 1975 when tasked to find ways to improve software development productivity in our organization. The first experiment involved two-programmer teams, which later became pair programming teams. The experiment was based on the idea that “two heads are better than one.” We accomplished a 125% gain in productivity in spite of doubling the number of people assigned to the effort, and also produced a 3-order decrease in errors. Chuck Tonies, my manager, and I developed a conceptual model of the development environment that was published in Software Engineering (1979) as an Effectiveness Formula based on three attributes: communication, management quality, and applied technology. Each attribute value ranged from zero to one. The organization effectiveness was a product of the three attributes. Our initial goal was a description of the environment, not a tool for productivity improvement even though the formula could forecast a development productivity.

My second task was the development of a software development cost and schedule estimation model (Seer), which is one of the most widely used in software development management today.

During the following years I was directly involved in predicting cost and schedule for numerous large software developments contracts both in and out of the government, and assessing the capability of organization to meet stated productivity goals. I found the conceptual formula made it possible to quite accurately predict organization productivity as early as the pre-contract assessments. I modified Seer (Seer II) in 1995 to allow the model to directly project an organization’s native capability in support of the Effectiveness Formula, and the ability to map the estimating capability to the Effectiveness Formula. Seer II occurred at the same time as the rapid growth of Agile methods in software development which further validated the conceptual model.

It was 2005 before I was comfortable with the validity of the formula, the underlying data, and especially the lessons I learned working with organizations to improve their development environments. The data obtained from Agile developments mapped remarkably well to the formula and showed a significant difference from the results obtained from traditional developments. It was time to write the book!

InfoQ: You stated that the software industry has only reached limited results in real productivity gains. Can you give an indication how big the improvement actually is?

Randall: I have collected software development data from all kinds of projects completed from the early 1960s through today as a basis of the cost and schedule estimating work that I have been involved in since the mid 1970s. The data is primarily from traditional developments, although there is enough data from “inspired” developments to include in the overall data. The data begins with Assembly language projects with no tools, nor modern methods to support them. Almost all of the improvements in tools, languages, and methods have occurred in the technology attribute of the Effectiveness Formula. We have seen no improvements in the communication attribute unless we want to include the cubicle as a communication improvement. Management style has drifted very little beyond the five functions of management specified in the early 1900s. The rate of productivity improvement, with some flurries and dips, have progressed at the rate of about 1.5 source lines per person-month of effort (sloc/pm) per year since 1960 due entirely from technology gains. Putting that information in another form, productivity of 60 sloc/pm in 1960 equates to an average productivity of about 140 sloc/pm today. This data is based on source lines written, and does not account for the 3:1 gain shifting from Assembler to Fortran, or the power of the visual languages. This is based on the raw number of source lines produced.

InfoQ: What in your opinion limits productivity gains?

Randall: To sum the lack of productivity gains in just a few words – Ignoring the communication and management quality attributes of the formula. Traditional development organizations today look and act like the same organization we worked with in 1970 except for the technologies available today.

The absolute isolation provided by the cubicle eliminates communication and collaboration.

Communication consists of three parts: visual, aural, and written. By removing the visual and aural elements, the communication is limited to only 7 percent of the original. Communicating with the programmer in the next cubical through the network is actually no different than reading the written word. You can send a question (email = another set of written words) and a response (also email) arrives as written words. We can further limit communication by failing to provide areas where programmers can communicate face-to-face. Isolation also reduces motivation when the people are not able to function as a group.

Management quality, according to the five management functions, are defined as: planning, organizing, commanding, coordinating, and controlling. These functions define traditional management, but they do not deal with the people issues that affect productivity. These issues, defined in the 1920s and known as the Hawthorne Effect are virtually ignored in traditional management. Managers are still isolated from the developers in traditional organizations. I describe traditional managers as directors, and modern managers as leaders; or in other words, sheepherders and shepherds.

What amazes me most is there have been well-publicized successes in the system development fields where the traditional motive have been discarded and the organization run like a team. For example, the Lockheed Skunk Works began in the mid 1940s and demonstrated the productivity that could be achieved with teamwork. Imagine a P80 aircraft in 247 days. I have seen many good project leaders taken offline because their way is not the way the organization does business.

InfoQ: Can you share your thoughts about how the rise of agile software development has contributed to productivity improvement?

Randall: Agile software development has certainly proven the importance of communication and collaboration in development. Agile has been with us for almost 20 years (or longer if you consider the 1975 experiment with pair programming) with essentially no impact on traditional development methods. The cubicles are still there and management is still hung up on CMMI. Agile development validated the Effectiveness Formula and the importance of people in the productivity model. The agile focus on individuals and interactions over processes and tools clearly states the importance of people in the model.

InfoQ: Together with Chuck Tonies you defined an effectiveness formula which describes the value that engineers can contribute. Can you elaborate on that?

Randall: Chuck and I defined a simple conceptual model of the software development environment based on three attributes of the environment: communication, management support, and development technology. The concepts of motivation, collaboration, and development teams are encompassed in the attributes.

While gathering the material for the book, it became obvious that the model applied to almost all development environments. The Lockheed Skunk Works is only one example of the model’s implication in system development environments. The many Agile methods all utilize modern management techniques and effective communications.

Pushing a little further, the elementary education classroom is another environment involving students, teachers, and technology. Think of the effectiveness improvement if the students were allowed to collaborate with close support from the teachers.

InfoQ: You described an experiment with pair programming in your book. Can you tell how it was done? What were the benefits of working in pairs?

Randall: When I was asked to find a way to increase the organization productivity, I fell back on two concepts: (1) Two heads are better than one, and (2) the experience I had working as a team struggling through a EE undergraduate program. The experiment platform was a multitasking system integration program of about 50,000 source lines to be written in Fortran. There were six independent tasks to be developed.

The organization provided 10 programmers for the development with experience levels ranging from recent college graduates to an experienced system programmer. Five teams were assembled pairing the most experienced with the least experienced , and the fifth team with two of roughly the same experience level. Each two-person team was assigned to an office in near proximity to the other teams. The team was to work together with one workstation, one member acting as a “driver” with the 2nd member acting as “navigator” or “observer.”

The project manager was a hands-on leader. I think of him as a perfect shepherd who had to work through the rough parts of getting each team to collaborate. None of these teams had any prior experience in a team environment. The team that was comprised of the senior programmer and his team mate, just returning to work after 6 years of raising a child, were an issue initially. He thought she should get the coffee and pick up the reports from the printer. He would do the thinking. After a couple of serious discussions, the senior programmer found that his partner was quite talented.

System integration was the time that the proof of the team idea became obvious. When the first two tasks (about 10,000 source lines) came together, there were only two design errors to resolve. This integration was between the components of two independent teams. The third task (about 5,000 source lines) was integrated with zero errors. There were a few errors uncovered with the last two tasks, but the overall result was phenomenal. They achieved a 125% productivity gain and a very significant decrease in errors.

The most memorable part of the experiment came at the end of the project. The results obtained from the experiment were presented at an organization staff meeting. The response from the other project managers was that we could not adopt this new approach because the senior programmers in the organization would leave rather than work as part of a two-person team. My thought was “Let them leave. We would be better off.” The project manager did transfer out of project management, and became the head of a computer center.

InfoQ: In your book you wrote about positive effects from working with teams. Can you name some of them?

Randall: The benefits of working as teams are: motivation, problem isolation, brainstorming, communication, continuous walkthroughs, collaboration, and more. What I observed was a repeat of my college experience in collaborating on homework and test preparation.

InfoQ: In your opinion do these positive effects from teams explain the productivity gains that can be reached using agile software development?

Randall: I certainly do. Pair programming is one of the Agile approaches. Agile, in general, is team based. You see that in the agile focus on individuals and interactions over processes and tools clearly states the importance of people in the formula. Agile development is largely a team approach to problem solving. Each approach has somewhat different characteristics, or formalities, but motivation, communication, and collaboration are keys in all of the approaches. There is an adage I think about a lot. Every project has problems, and they are all people problems. The 1920 Hawthorne Experiment is one of the first studies demonstrating the importance of people in productivity improvement.

InfoQ: There are many factors that influence productivity. Doesn't that make estimation models like SEER rather difficult to use? Are there any lightweight solutions?

Randall: There are about 30 parameters that must be considered when making a project estimate because of the different factors the influence to project, such as memory constraints, requirement stability, development system experience, etc.

When looking at an organization’s underlying productivity without the project constraints, there are only a set of 7 parameters that can be established with only an introduction to the organization. If you are concerned with only your organization, you probably already know the correct values to establish basic productivity.

Herein lies the problem. You must honestly set the values of the parameters. When you start with the assumption that your organization ranks at about the 75th percentile of all organization, you can lie to yourself all day long. Assessing each parameter realistically is the hard part.

You might call this a lightweight method that I use for quick estimates, and it is a cross check I often use to validate a productivity assessment is to find the productivity achieved during the last project. It will probably be close to that of the next project; that is, unless something is changed prior to the next project. If I find the productivity estimate is 25% higher than the last project, you cannot accept “We will be 25% smarter than we were last time.” That is only wishful thinking.

InfoQ: Your book mentions that knowledge retention is a major factor for the productivity of software maintenance teams. What can organizations do to increase their abilities for retaining knowledge?

Randall: Most software development estimating models only include factors to account for (1) corrective maintenance, (2) adaptive maintenance, and (3) perfective maintenance. All three of these factors assume that some change(s) will be required to the maintenance effort. Block changes are typical of these types of charges. Sage, based on Seer II is the only estimating package that assumes that no changes may be required to the operational software.

For example, let’s assume we have just inherited a 1,000,000 source line operational radar system. We have no prior knowledge of the software. It is easier to visualize the maintenance effort if we picture the source software is contained on punched data cards. You do remember those days, I hope. One million source statements require 500 boxes of cards. How many software engineers are required to understand and repair that many boxes of cards? The effort is outside the normal maintenance changes to the software. Do we have to support 5 or 50 people to retain knowledge of the system? We cannot afford to assume that if it breaks, we can think about instantly learning the system with the few engineers we projected to make corrective changes. Knowledge retention is the bucket we have to add to the effort to keep the system working. Annual change traffic is usually far short of the requirement. By the way, an average organization requires about 40 engineers to deal with system knowledge in this example. That is about 12.5 boxes (25,000 source lines) per engineer in the real world. I know of no shortcuts to increase the amount of memory (mental) that an individual can devote to that size of a task.

InfoQ: In your book you stated four commandments for effective communication. Can you name them and give some advice for organizations that want to adopt these commandments?

Randall: The four commandments are logical and real. The first commandment states the organization shall do nothing to create barriers that limit communication. The typical barrier is the infamous, but common, cubicle. Teams work better in open spaces where movement is relatively unlimited.

The second commandment says do not put two or more teams in the project area. People not related to the task at hand are also communication barriers, and the presence of these outsiders creates noise and lowers motivation.

The third commandment says give the development team the whiteboard, conference tables, markers, and popcorn needed to communicate freely.

The last commandment says do not try to share team members between projects. That doesn’t help free communication and motivation at all.

The agile world has already found these commandments, or guidelines, are necessary to keep the ideas and work flowing smoothly.

About the Book Author

Randall W. Jensen, Ph.D., is a software acquisition consultant with more than 40 years of practical experience as a computer professional in hardware and software development. For the past 30 years, he has actively engaged in software engineering methods, tools, quality software management methods, software schedule and cost estimation, and management metrics. He retired as Chief Scientist of the Software Engineering Division of Hughes Aircraft Company’s Ground Systems Group, and was responsible for research in software engineering methods and management. He developed the model that underlies the Sage and the Galorath, Inc.’s SEER-SEM software cost and schedule estimating systems. Jensen received the International Society of Parametric Analysts Freiman Award for Outstanding Contributions to Parametric Estimating in 1984. He has a doctorate in electrical engineering from Utah State University. He can be contacted at


Rate this Article