Ken DeLong described why he thought software development was especially HARD:
Everyone knows that writing software is hard. But I want to talk about why software development is hard.
Software development is afflicted by several qualities that push it into the realm of complex adaptive systems - and often push it into chaos (or close enough). These qualities are
- nonlinearity
- feedback
- time delay
- variation
This is not a novel observation, although many of us in the field meet people who are oblivious to this fact. Perhaps some of us have heard it and didn't completely understand what this means. That's where Ken's blog entry is particularly helpful:
During rush hour, the road is packed, and everyone may be driving fairly quickly. But then some guy rubbernecks, and slows down to 30 mph, just for a few seconds. The result? A massive traffic jam. Ever sit in bumper-to-bumper traffic, only to suddenly zoom up to 70 mph? What's up with that? Why was there a giant backup if there was no accident or other obvious cause? Earlier, there was some variation - someone rubbernecked - the nonlinearities compounded, and the jam formed. Many of these complex systems tend to be self-sustaining, and the traffic jam appears to be one of them. Eventually it unwinds, but far, far slower than anyone would guess based on intuition.
The events that happened from one guy slowing down - no accident at all - caused the whole mess! So what's the point? Software is hard and dangerous and the slightest thing can throw it off. What else?
One of the ways to avoid huge traffic jams on a constrained capacity highway is to use metering lights - keep the utilization of the system at a level where normal amounts of variation will not be amplified indefinitely. This is the same way we keep our CPU usage on our production servers at 30% or less so that we can handle spikes in traffic without thrashing the servers.
Therefore, there must be some feedback in the system so it can handle variations without coming to its knees:
In Agile, the number of story points accepted for a sprint is adjusted so that everything on the Sprint can get to the Done state within the Sprint. Metering lights.
Another way to look at the adjustment, the metering lights, in Agile software development is described by Jurgen Appelo in 10 Principles of Agile Project Time Management:
1. Use a Definition of "Done"
2. Use Timeboxes to Manage Work
3. Don't Add Slack to Task Estimates
4. Defer Decisions
5. Reduce Cycle Time
6. Keep the Pipeline Short and Thin
7. Keep the Discipline
8. Limit Task Switching
9. Prevent Sustained Overtime
10. Separate Urgency from Importance
Jurgen's goes into more detail with each of these ten principles and gives references for each one for in-depth reading. Software development is Hard. One of the main reasons is that it is a complex adaptive system. Agile - when done right - seems to do a very good job of providing stabilizing feedback.