Holding a Program in Your Head

Posted by Kurt Christensen on Oct 19, 2007 10:00 PM

Community

Agile

Topics

Agile Techniques ,

Collaboration

Tags

Antipatterns ,

Best Practices ,

Complementary Practices ,

Criticism

In a recent essay titled "Holding a Program in Your Head", Paul Graham - essayist, Lisp hacker, and the creator of Yahoo! Stores - argues that "your code is your understanding of the problem you're exploring. So it's only when you have your code in your head that you really understand the problem." Unfortunately, as every programmer knows, this is easier said than done:

It's not easy to get a program into your head. If you leave a project for a few months, it can take days to really understand it again when you return to it. Even when you're actively working on a program it can take half an hour to load into your head when you start work each day. And that's in the best case. Ordinary programmers working in typical office conditions never enter this mode. Or to put it more dramatically, ordinary programmers working in typical office conditions never really understand the problems they're solving.

So what can a developer do to get a program loaded into their head? Graham offers eight suggestions:

1. Avoid distractions
2. Work in long stretches
3. Use succinct languages
4. Keep rewriting your program
5. Write re-readable code
6. Work in small groups
7. Don't have multiple people editing the same piece of code
8. Start small

Agile processes and practices can be viewed as a mapping of what happens organically in a successful startup company to what can possibly be done within a large organization. As a partner of the seed investment firm Y Combinator, Paul Graham targets most of his advice towards small startup companies, and so the question becomes which of these best practices map to agile practices. Certainly most agile developers are accustomed to writing re-readable code, continually refactoring, working in small groups, and starting with the smallest thing that provides real value. And many agile developers are increasingly moving towards more powerful languages such as Ruby, Erlang, Haskell, or even Common Lisp.

But what about those suggestions that don't easily map to agile practices? (1) and (2) go hand-in-hand, and some would argue that working in a shared lab space is the antithesis of avoiding distractions. Another common agile practice is shared ownership of code, contradicting (7). So do the agilists have it wrong? Or does the conflict between these recommended practices reflect the unavoidable differences between working for a small company and working for a large company?

1. Back to top

   Mental Models, abstraction and communication

   Oct 21, 2007 2:19 AM by Paul Oldfield

   Well, I was all prepared to disagree, but I like a lot of what is said.
   
   My first and most important point is that this approach does not scale. If you want an approach that scales, you need to be able to have one or more levels of abstractions, you need to be able to separate the concerns and hide the details behind interfaces. Your mental model will not be in code.
   
   Let's go through some of the points he raises.
   
   Avoid Distractions: definitely not very agile. I favour Alistair Cockburn's description of 'osmotic communications' - one has a subliminal awareness of what's going on in the rest of the team. This isn't a distraction until it needs to be; until something comes up that needs your input.
   
   Work in long stretches: I favour sessions about 90 minutes long. I really like and agree with Paul G's point that time away from a problem can give one's subconscious a chance to solve it. To me, wanting longer sessions is a smell; it says to me that the mental models are not adequate. For me, a good model is an aide-memoir; one glance at the model and the whole thinking behind it comes rushing back into my mind. Sure, if I haven't solved a problem then it takes a while to get back into the train of thought - but that tells me the problem is not being cut down into small enough chunks.
   
   Succinct Languages: Agreed.
   Keep re-writing: Only to improve the code.
   Re-readable code: Essential.
   Work in small groups: I favour group ownership of the code. This scales better and means that you never lose the ability to do all-encompassing re-designs.
   
   Several People Editing: His suggestion here is definitely not agile. It sounds quite dysfunctional. The team should be working to a shared vision.
   
   Start Small: Agreed, but the problem he addresses is better solved by developing abstract models that keep the big picture in mind, while delving into the detail one area at a time.
   
   In general, these practices talk to me of poor team communication, poor abstract modelling. If you're listening, Paul, take this as hints as to how you might improve, not as destructive criticism.

   Reply

2. Back to top

   Re: Mental Models, abstraction and communication

   Oct 21, 2007 2:35 PM by Kurt Christensen

   As always, thanks for your thoughtful comments. I like most of what Paul Graham has to say, which is why I always enjoy trying to map it to practicing agile in a large IT shop. It works more often than may be obvious, but not always.
   
   Which brings me to your comments. I agree with what you said, but I don't think it's because he's wrong, I simply think it's because he's addressing groups of 2 or 3 incredibly talented developers working on apps for startup companies. In that case, there are different forces at work, and - more importantly - the codebases are new and small. As an example, Reddit was built on top of web.py which is all of a thousand lines of Python code. Contrast that with one of my current clients, which has about 2 GB of source code feeding into a single product suite. I don't think the values in those two organizations necessarily differ, but the practices certainly will.

   Reply

3. Back to top

   Re: Mental Models, abstraction and communication

   Oct 22, 2007 3:52 AM by Paul Oldfield

   Hi, Kurt.
   
   Yes, I'm all in favour of applying what works best in the given circumstances. I don't subscribe to the view that there are *any* universally best practices. On the other hand, a successful start-up will grow into a different set of circumstances. It makes sense to be aware of the problems that will come to the surface when that happens.

   Reply

4. Back to top

   Re: Mental Models, abstraction and communication

   Oct 22, 2007 8:14 AM by Deborah Hartmann

   > Avoid Distractions
   
   I've worked in Agile environments where "shared workspace" led to distractions, because of:
   
   - poor inter-team boundaries (low walls, with team A overhearing team B's discussions: a real annoyance because team B's "noise" didn't contribute to team A's work, and vice-versa). This was resolved by adding a full wall in that location.
   
   - poor teamroom flow (everyone sitting facing *out* at the boundary wall of the team room. Every conversation was shouted across the room). This was resolved by moving the tables away from the walls, to create smaller "pools" of conversation.
   
   In both cases, space design contributed to chaos, rather than enhancing it. As long as the space itself is agile (susceptible to change, as needed) this just becomes part of the continuous improvement cycle.

   Reply

5. Back to top

   Re: Mental Models, abstraction and communication

   Oct 23, 2007 2:00 AM by Paul Oldfield

   Agreed. We want to distinguish between unproductive distractions and productive distractions. I wrote about productive distractions several years ago:
   <www.aptprocess.com/whitepapers/risk/RiskToPatte...>
   There's also a page on that site titled "Interrupts Unjam Blocking" (the name says it all!).
   We usually consider these ways of thinking to be already embedded in the ways an agile team works.

   Reply

• Older >