Software Development: A Traffic Jam Waiting To Happen
Ken DeLong described why he thought software development was especially HARD:
Everyone knows that writing software is hard. But I want to talk about why software development is hard.
- time delay
This is not a novel observation, although many of us in the field meet people who are oblivious to this fact. Perhaps some of us have heard it and didn't completely understand what this means. That's where Ken's blog entry is particularly helpful:
During rush hour, the road is packed, and everyone may be driving fairly quickly. But then some guy rubbernecks, and slows down to 30 mph, just for a few seconds. The result? A massive traffic jam. Ever sit in bumper-to-bumper traffic, only to suddenly zoom up to 70 mph? What's up with that? Why was there a giant backup if there was no accident or other obvious cause? Earlier, there was some variation - someone rubbernecked - the nonlinearities compounded, and the jam formed. Many of these complex systems tend to be self-sustaining, and the traffic jam appears to be one of them. Eventually it unwinds, but far, far slower than anyone would guess based on intuition.
The events that happened from one guy slowing down - no accident at all - caused the whole mess! So what's the point? Software is hard and dangerous and the slightest thing can throw it off. What else?
One of the ways to avoid huge traffic jams on a constrained capacity highway is to use metering lights - keep the utilization of the system at a level where normal amounts of variation will not be amplified indefinitely. This is the same way we keep our CPU usage on our production servers at 30% or less so that we can handle spikes in traffic without thrashing the servers.
Therefore, there must be some feedback in the system so it can handle variations without coming to its knees:
In Agile, the number of story points accepted for a sprint is adjusted so that everything on the Sprint can get to the Done state within the Sprint. Metering lights.
Another way to look at the adjustment, the metering lights, in Agile software development is described by Jurgen Appelo in 10 Principles of Agile Project Time Management:
1. Use a Definition of "Done"
2. Use Timeboxes to Manage Work
3. Don't Add Slack to Task Estimates
4. Defer Decisions
5. Reduce Cycle Time
6. Keep the Pipeline Short and Thin
7. Keep the Discipline
8. Limit Task Switching
9. Prevent Sustained Overtime
10. Separate Urgency from Importance
Jurgen's goes into more detail with each of these ten principles and gives references for each one for in-depth reading. Software development is Hard. One of the main reasons is that it is a complex adaptive system. Agile - when done right - seems to do a very good job of providing stabilizing feedback.
Avoid Congestion, keep distance
If you're going to use a congestion metaphor, I'd suggest defining what that means. Congestion=bottleneck. Something in your team structure is letting some people get idle while others are dealing with a problem. If this occurs, you can look at a few things
1) How much specialization is there in your team? If you're getting bottlenecks, is it b/c you don't have enough cross functionality? Get some training and knowledge sharing cooking!
2) How many tasks have other tasks as prerequisites? Do you have enough of the chains for someone to pick up on if they get stalled on another task chain b/c of someone else? Are your features divided into tasks in the right way for your team and project? Can they be sliced and diced differently?
3) The dead simplest...Are you allowing enough time for tasks? Realistically, if you constantly are seeing bottlenecks/congestion, you may just be underestimating certain types of tasks or not planning on varying speed to check in (and remember, the "slower" developer may be slower b/c they are producing better quality, so look at things like that)
And finally, my favorite thing to harp on in infoq...
4) Do you have people putting in too many hours? Is someone stalled simply b/c they feel they should be working 70hrs a week while someone else is working "normal" hours? Conversely, is the coder who is always the bottleneck also the one putting in all the hours? If so, you've got a powder keg. If it's not already hurting you, it almost certainly will (and mandatory code reviews for that person are not an option!)
software is hard because it is complex
Re: software is hard because it is complex
I would disagree with the basic premise - software is hard because it is complicated - not complex. If it were truly complex, it would be easy! I am about 2/3 of the way to completing a book making the "easy" argument in detail. Perhaps some preview infoQ articles if the editors find them of interest.
According to M-W.com:
complex: hard to separate, analyze, or solve
complicated: difficult to analyze, understand, or explain
If you understand complexity as to describe the state of an arrangement and complicated the level of difficulty of a situation or task then I would assume both apply to software development.
Re: software is hard because it is complex
In everyday usage, we often use "complicated" and "complex" interchangeably. Which makes it a little bit hard to discuss complexity. I mean, the "good" kind of complexity that you find in nature, for example.
We could say that we need the scientific definition of complexity, but science has unfortunately come up with at least 32 different definitions, that don't agree with each other. But if we cut through the confusion a bit, something like this would be workable:
Complicated is when something contains many intricately combined parts. It is something that is hard to figure out. Even if you do figure it out, there's no guarantee that things are put together in a sensible way.
Complex is when something acts as a system, and it is exhibiting systemic properties that aren't obvious. It is something more and different than simply a sum of its parts. There might or might not be many parts, but the result is something not very transparent, which takes on a life of its own in some fashion.
An Airbus A380 is complicated. A jellyfish is complex. The Paris Metro network is complicated. How people use it is complex. Your skeleton is complicated. You are complex. A building is complicated. A city is complex.
This description is from Flemming Flunch's blog.