An Evolutionary Perspective of Software Development

Posted by Amr Elssamadisy on Jun 29, 2009

Sections

Process & Practices

Topics

Agile Techniques ,

Agile in the Enterprise ,

Adopting Agile ,

Agile

Tags

Adoption

Memes, originally coined by Richard Dawkins in The Selfish Gene are cultural genes; ideas that propagate among people and affect the way we think and act.  Julian Everett has suggested that we can look at software development practices, ideas, and culture as a collection of memes.  By doing so, our understanding of what works and why can be turned on its head.  Questions like "are IT Projects the correct way to maximize business  value for an organization?" and "Is the XP idea of Onsite Customer wrong?" are raised and lead to surprising answers.

Julian starts out describing the meme lifecycle:

1) Uncertainty: resulting from an unsolved problem or a phenomenon that does not fit within our current worldview.

2) (bio)diversity: a range of new ideas are then considered as possible solutions or ways of integrating the new experience into our worldview.

3) Dominant Meme: one particular idea is adopted as the “best” explanation/solution, and becomes embedded into the updated worldview.

4) Zombie Meme: social or business context changes, making the previously dominant meme no longer relevant. It is now “dead” and needs be laid to rest before it starts causing havoc.

5) Lazarus Meme: a dead idea that is brought back to life, because changes to a social or business context suddenly make it relevant again.

Then Julian describes how this meme lifecycle can be seen in business:

Within the context of business, memes can be seen as the idea-pool DNA of an organisation that get expressed as an “extended phenotype” or sliding scale of effects: from commercial strategy to budgets, marketing, IT projects, etc.

If we consider the subset of memes that constitute well-factored MMFs [A minimum marketable feature (mmf) is the smallest possible set of functionality that, by itself, has value in the marketplace. You could release your software with just one of these features and you would see some benefit.], then real options are the mechanism for determining what organisational memes should be expressed and when (via the optimal exercise point), and equally importantly which memes need to be culled (via negative valuation).

Zombies are the things that cause the real damage: financially we are at present arguably living through the fall-out from a economic zombie meme – the efficient market hypothesis (which resulted in deregulation of capital markets, etc). Closer to home, some of the worst problems I have seen caused by technology programmes has resulted from “zombie projects”, where there has been a resistance to terminate despite clearly changed business conditions and commercials that no longer add up…

Now, if we were to look at the units of business value as an MMF, then the idea of IT Projects becomes obsolete and wasteful:

Evolutionary theory changed the focus of biological thinking from organisms to genes. In the same way, an evolutionary perspective to software delivery emphasises the need to stop focussing on IT projects and instead think solely in terms of MMFs. Then organisations with high resource liquidity become business value hunter/gatherer cultures, where teams are continually redeployed to whichever part of the company currently has the highest value MMFs to be implemented.

 

Julian also aims and takes fire at the XP notion of a customer by looking at the practice as a meme:

Let's take the example of XP Customer again. I would (contentiously) suggest a significant reason that meme spread through the software industry environment was because the majority of people who formulated XP were consultants of some sort, i.e. XP Customers were also their customers in a real, financial sense. In other words, the economic reality of the fitness landscape favoured a definition of customer informed by "project success", but where that success was the _consultant's_ rather than the client's. And as an IT consultant what's the best way of ensuring you get paid? The old meme was "on-time, on-budget" used as a standard charging vehicle, but that then got superseded by the even fitter notion of the happy, "omniscient" XP Customer. Completely understandably, because as a consultant you wouldn't want a financial dependency on a client's possibly unsound wider commercial strategy (hence the difficulties with agile contract definition). However from a client perspective, stakeholder satisfaction is largely irrelevant in the big picture - it is all about commercial strategy and delivering long term ROI.

 

This is a compelling model we can use to view the world of software development. What are your thoughts?

1. Back to top

   Addendum

   by Amr Elssamadisy

   Although Julian Everett has explored the suggestion more fully,
   especially with respect to lifecycle, he's not the first person to
   suggest thinking about software practices in terms of memes. Kevlin Henney
   presented the idea in relation to software patterns in 1996, using the
   term "memetic engineering" in that context:
   
   www.two-sdg.demon.co.uk/curbralan/papers/timele...
   
   So, in a sense, this work is a Lazarus meme also.

   Reply

2. Back to top

   Shallow Understanding?

   by Nik Boyd

   Julian's sarcastic characterization of the XP Customer as "omniscient" smacks of cynicism, devaluation, and a shallow understanding of the tenets that motivate XP. This is also reflected in his characterization of the idea of an XP Customer in opposition to the "on-time, on budget" idea. Clearly, these ideas needn't be in opposition, but the definitions matter, and they are very project specific.

   
   
   

   But, what is a project anyway? Is it just (potentially) a Sprint or Iteration, or is it something larger (i.e., a Release)? It seems like the idea of a "project" needs some further clarification. Perhaps it has something to do with stable product concept(s), relative team continuity, supportive infrastructures (both technical and business), etc.

   
   
   

   None of this stuff evolves in isolation. There is actually an ecology of ideas and real-world people, places, and things. Perhaps a deeper examination of the related concepts is in order. I recommend something like Cockburn's Agile Software Development as a good place to start.

   Reply

3. Back to top

   Re: Addendum

   by Chris Matts

   Hi Amr
   
   I agree that "meme" has been a popular term for many years. As such, Julian is not the first to use the term. What is different here is that Julian has mapped "meme" to that the "gene" framework. It makes the idea adoption process much clearer and explains why some work and some fail.
   
   I had a sneak preview of his Agile 2009 presentation last night. I suspect that Julian's session will be one of the "finds" of the conference.

   Reply

4. Back to top

   Re: Shallow Understanding?

   by Julian Everett

   Hi Nik,
   
   Thanks a lot for your feedback. My comments were absolutely not intended to devalue the motivations of XP. They were made within the context of a discussion on the real options group where I was trying to highlight the need for close pairing between analysts/architects and product managemers in order to help product management "customers" make better informed prioritisation choices - so the people spending the money can manage their technology investment risk much more effectively. Recently we have rightly been quick to criticise investment banks for their misguided risk management strategies, however from my experience the techniques used to manage technology investment in software delivery programmes are frequently even worse. I have found it helpful on IT projects to ask the question: would I want a fund manager to be using these risk management tools to manage my pension? Real options-based projects are the only instance where the answer to that question has been "yes".

   Reply

5. Back to top

   Re: Shallow Understanding?

   by Chris Matts

   Nik
   
   I think Julian's characterization of the XP Customer as "omniscient" is neither cynical nor sarcastic, but a reflection of view stated by some XP thought leaders. Ron Jeffries recently stated on the Kanbandev group that XP assumes "a customer that knows everything about the business and exactly what they need...." ( paraphrase ) which sounds pretty "All Knowing" or "Omniscient" to me.
   
   Your question related to projects as sprints or iterations indicates an "IT" view of projects. Julian is looking at this from the perspective of the investor in IT.
   
   If you would like to join in the conversation, please join us at finance.groups.yahoo.com/group/real_options_dis...

   Reply

6. Back to top

   Re: Addendum

   by Kevlin Henney

   It was Dawkins that mapped memes to the framework of genes :^) Julian has focused on the lifecycle of ideas from that perspective. That is the novel touch. The correlation to pattern concepts and lifecycles is surprisingly strong, but the particular perspective here is fresher and offers a different angle.

   Reply

7. Back to top

   Re: Shallow Understanding?

   by Amr Elssamadisy

   Respectfully Nick, you should not dismiss an idea unless you've taken a sincere look at what it proposes.
   
   As I understand it (and I could be wrong), saying that the XP Customer meme propagated because most early conveyors of this idea were consultants is a real possibility. The whole idea behind the XP Customer is to get closer to value (to the customer). The fact that a meme catches on is not a moral judgment on the persons, but an explanation why it is attractive.
   
   The XP Customer is an improvement, but as many of us know, there are significant improvements that can be made especially when you have a less-than-omniscient customer.

   Reply

• Older >