Programming Processes

In "The Secret Curse of Expert Archers", New York Times columnist Katie Thomas describes the strange affliction known as "target panic", which causes world-class archers to inexplicably lose the ability to control when they release an arrow at a target. While usually regarded as a psychological problem, new research indicates that target panic may actually be a neurological disorder.

Neurologists are beginning to look to processes deep inside the brain in order to understand afflictions which have previously gone unexplained. For example, in a recent article for The New Yorker, Dr. Atul Gawande describes a new treatment for "phantom limb pain" - the acute pain felt by amputees in arms or legs which do not exist. A doctor places an amputee in front of mirrors which create the illusion of a full set of limbs, and then has the patient perform a variety of tasks - for example, conducting an imaginary orchestra. A new study from researchers at Walter Reed Hospital shows this "mirror box therapy" to be remarkably effective in treating phantom limb pain. Astoundingly, it appears that providing a new and unexpected set of sensory inputs to the brain modifies the processes within the brain by which those inputs are handled.

Of course, in the world of artificial intelligence, there's nothing astounding about the idea of using data to modify the algorithms which operate on that data - such "learning algorithms" are used in applications ranging from speech recognition to credit card fraud detection. In fact, as it becomes possible to work with increasingly large data sets, it would appear that the data given to most learning algorithms is more important than the algorithms themselves. In a presentation given at Startup School 2008, Peter Norvig described the performance difference between five self-modifying learning algorithms for natural language processing, and showed that the performance improvements gained by selecting a better algorithm are rarely as great as the performance improvements to be had by simply processing more data.

But could this be a metaphor for the process of developing software? In the book "Metaphors We Live By", George Lakoff and Mark Johnson describe how metaphors shape understanding, and the ways in which metaphors simultaneously reveal and obfuscate the world around us. For the brain, and for software, and for software development, the dominant metaphor is that of a machine - in other words, hardware. But in both cases, better progress might be had by instead viewing processes as software - programmable; prone to error, but easy to fix. If software is to be the metaphor for software development, then software development processes should be created and refined in the same way as software should be created and refined - loosely at first, building only as much as needed, and always iteratively and test-driven.

Although it can be frightening to abandon the idea of prescriptively defining optimal processes, the reality which seems most consistent with both the human brain and with software itself is that the best way to build a software development process for a particular group of people is to forget about process definition and focus on process evolution.