Tracking change and innovation in the enterprise software development community
QCon London 2010!
March 10-12, 100
Speakers, 80 sessions,
"THE" team leads and,
architects conference.
Scaling Scrum
in the enterprise?
ScrumWorks Pro
can help.
Start a trial today
Free Book Offer
SOA Demystified
$40 Value
Posted by Mark Figley on Jul 24, 2007
Carl Lewis recently found an old but still interesting piece by Dennis Forbes about the tradeoffs of code reuse. Lewis elaborated a bit on one of the more controversial concepts from Forbes' blog entry: that contrary to conventional wisdom of code as an asset, code has very little value outside the immediate context of the organization that created the code in question. Forbes goes on to assert that if the code asset has not been explicitly engineered for generality, code reuse even within the same organization is also problematic when reuse is attempted across projects.
Forbes attacks the tendency of many in-house dev teams to spend a lot of time building in-house frameworks and libraries intended for reuse across projects:
The greater the bulk of code you accumulate, the more intrinsically you tie yourself to your current developers (and the more they occupy their brain with information that is only applicable in one organization or team), and the more difficult it will be to bring new developers online. Such frameworks and libraries often come with enormous learning curves for new hires - especially as documentation is virtually always ignored - and they can seldom be reused for anything else without significant refactoring (because they likely weren't truly designed for reuse, or they were in only the most arbitrary and superficial of ways…
… The question every organization needs to ask itself, then, is what value they could sell their "reusable code" for - what, realistically, would competitors and new entrants in the field offer for it? The answer, in almost every case, is $0, and they wouldn't want it even at that price. There is extraordinarily little code theft in this industry (even though we're in the era of burnable DVDs and USB keys) because most code - above and beyond the industry-wide frameworks and libraries - has no value at all outside of a specific project with a specific group of developers. Trying to use it for other projects is often worse than starting with nothing at all.
Forbes concludes that because properly abstracted and generalized frameworks are more expensive to design and develop than one-off solutions, and since the time and cost for a developer to absorb the complexity of a new framework is rarely taken into account when doing a code reuse cost benefit analysis, it makes sense to reuse code a lot less often than one would think. It is notable that Forbes is not against reusing code and frameworks altogether; he is much more positive about reusing code in industry standard libraries for instance. In fact, Forbes advocates the adoption of broad industry-standard frameworks (such as open source) that by necessity have the proper abstraction and encapsulation boundaries built in as an alternative to complex internally developed frameworks.
In Carl Lewis' post on the subject he highlights Forbes' $0 value assessment of most codebases. Lewis shares just one example of when he has encountered an organization that has an inflated sense of the value of their code, calling them “jealously protective” of that code. Predicatively, when Lewis flew thousands of miles to be onsite so that the code wouldn't have to leave the premises, Lewis found that the actual codebase being jealously protected was pretty bad. Due to examples like this Lewis believes that "in many cases, the risk of [code theft] happening is much less than is generally thought," and so companies shouldn't be afraid to share their code in cases where it may make sense.
And so Lewis and Forbes present an interesting reality check for us with regard to the value of our code assets, because as Lewis says:
It seems very counterintuitive that something that is so difficult and expensive to create can have so little value. I guess that's why so many organizations like to pretend that their code is much more valuable than it really is.
Unix, Linux Uptime & Reliability Increase While Patch Management Woes Plague Windows (Yankee Group)
The 5 Mandates of Software Development Teams - Presto Manifesto
Though agreeing when talking about general frameworks (and yes, this is still code reuse, it is just not developed and maintained by our organization), there is still space for code reuse over domains that differentiates our organization from the competition. These should be the domains where we should put our efforts, bearing in mind that we should decouple this code the most we can.
And watch out, lets not fall all over again on the NIH syndrome.
I have seen what Lewis describes on numerous occasions, and could not agree more. At the same time, there seems to be great value in the reuse that is happening in the open source community. How can companies emulate the effect of innovation from projects like Spring, Maven or Hibernate?
Secondary, even with the current level of technology, there seems to be a ramp-up on any project. In my experience, the hope is that code reuse will decrease the cost of this ramp-up. This, too, seems like an elusive goal. What can an organization do to improve the productivity of its projects?
It seems very counterintuitive that something that is so difficult and expensive to create can have so little value.
There are lots of things that were expensive and difficult to create that are worthless:
Ford Edsel
F111 Fighter-Bomber
F20 Tigershark
IBM PC Junior
New Coke
The article makes the point that most code has no value outside of the project it was written for, but within that project it can obviously be quite valuable. That's quite different from being worthless.
The article also makes the point that it is the context within which the code was developed that gives it its value. Phil Armour defined software as a media that holds knowledge in executable form. Given that definition, the context is essential to software's value.
That leads to the idea that in order to get reuse you need similar contexts.
I've given that about two seconds thought so maybe I'm all wet about the importance of context to reuse.
The problem behind nearly all framework projects that I have seen is that they have no consumer. The frameworks are conceived before any requirement for their use. Almost all of these projects I have seen fail.
In fact in one company being put on the framework project was being put in the unwanted pile.
Code reuse is vastly overhyped.
Certain industries may have a lot of IP tied up in some library or other - perhaps a pricing library - which will be reused across the company, but a library is not a framework.
I don't think that this article is saying anything that successful software leaders are not aware of.
I could go on about SSO & authentication "frameworks" - when pretty much everybody has AD.
I could go on about RPC "frameworks" - when everybody has webservices / corba
I could go on about messaging frameworks - when everybody has JMS / Rendezvous
I could go on....
I've seen a lot of waste.
I have seen what Lewis describes on numerous occasions, and could not agree more. At the same time, there seems to be great value in the reuse that is happening in the open source community. How can companies emulate the effect of innovation from projects like Spring, Maven or Hibernate?
If you look at the effort that went into Spring for example over the past couple of years, it's enormous. The resulting level of quality - that in the case of Spring is considered to be very high - is virtually impossible to achieve if it's not a core business.
There are literally very very few people that are as skilled and dedicated as Juergen Hoeller for example. For those few that are it's very unlikely that you can hire them.
So reaching the same level clearly can't be reached by any company (the notable exception may be Google, maybe). The best alternative to me is to contribute your code to for example Spring or any other framework. Accenture is about to do this in the case of Spring Batch. Google did it in the case of Hibernate.
These are very interesting examples of how you can take a good idea that has been prototyped in house and contribute it with the goal to increase it's quality.Secondary, even with the current level of technology, there seems to be a ramp-up on any project. In my experience, the hope is that code reuse will decrease the cost of this ramp-up. This, too, seems like an elusive goal. What can an organization do to improve the productivity of its projects?
The obvious quick win is to reduce the amount of code on a project, Grails has been designed with exactly that goal in mind.
As far as I'm considered there is currently room for a Rails/Grails like framework for messaging/web services; one whose first priorities are simplicity and performance.
I think code reuse inside an organization only makes business and technical sense when:
- it's a solution (that can be reused over multiple projects and solves a business need - for example, a payment solution)
- it's an algorithm (small maybe a function or class) that can be understood easily (compute check digits, alternative rounding, etc)
Everyhing else is hardly reusable, has documentation problems (usage examples, configuration), difficult to maintain, coupled to a technology in time (anyone for a framework in VB6?) and normally too inflexible (even big frameworks suffer from this).
code reuse is about progress and non-redundancy. Why waste developer cycles?
The argument that people get it wrong (no docs, too complex to use easily), and so it has no value strikes me as rather strange.
a better article would be "code reuse highly underrated, why do so many get it so wrong?". my opinion on that is that code reuse is often limited because of the human factor - people in organizations too often simply don't know what other groups are doing, and don't have informal communication lines between groups where they'd stumble upon this.
code reuse is about progress and non-redundancy. Why waste developer cycles?
The argument that people get it wrong (no docs, too complex to use easily), and so it has no value strikes me as rather strange.
a better article would be "code reuse highly underrated, why do so many get it so wrong?". my opinion on that is that code reuse is often limited because of the human factor - people in organizations too often simply don't know what other groups are doing, and don't have informal communication lines between groups where the y'd stumble upon this.
I disagree, the problem is that it takes a lot more effort, that may be not worth it, in order to make some code usable out of a specific context.
What if there was a way to search the code using something like Google for APIs or functions that look like they will do what we want. Haven't we all thought at one time or another "someone must have written this already!" (Anyone remember the days of pre-Java and pre-STL in C++ ?) Google Code Search and others already do this.
Now, what if, furthermore, we could use our tests (that we wrote first in our TDD practice), and run them against all of the returned searches to filter and prioritize? (This, as I understand, is also being done and will be available soon.)
So I guess what I am saying is that I agree that much of code reuse efforts seem to cause more trouble than they are worth. But I would jump on board to a code search engine that can use my tests.
My 2 piasters,
Amr
We've developed several reusable components at our company and yes, *gasp*, even an in-house application framework. These have worked for us and actually reduced the amount of work it takes to get a useable application up and running. It really depends on the types of projects that you work on. The key is to not try to solve all of the world's problems with what you are developing...otherwise it does take too long to develop and is usually worthless in the end.
It really depends. If you are building software for a family of printers, there are bound to be several places for code reuse.
Forget that - frameworks are all a type of code reuse - right? When was the last time you wrote anything with a user interface that didn't use a framework? How about an ORM tool? An inversion-of-control container? The list goes on and on...
Somewhere about 1990 I first came across the rule of thumb - you don't get a piece of code 'right' until it has been used in 3 different places.
Maybe we'd need to re-word that a bit, but I think it's a good guideline. First time the code is very specific to one context. Second time it suits two contexts. Third time it suits most contexts.
I believe TDD - or more specifically, writing the interface to be easy to test, helps in getting the code 'right for re-use' earlier. This drives us to write 'context neutral' code first time - though we still don't get that right every first time, of course.
Hi Paul,
believe TDD - or more specifically, writing the interface to be easy to test, helps in getting the code 'right for re-use' earlier. This drives us to write 'context neutral' code first time - though we still don't get that right every first time, of course.
Can you explain why TDD is better at this? I would hazard to guess that it would be worse because of Simple Design, and YAGNI making the solution specific to exactly one context. Even when we remove duplication, it is in one project not on many.
Does this sound right or am I missing something?
Amr
... TDD - or more specifically, writing the interface to be easy to test, helps in getting the code 'right for re-use' earlier. This drives us to write 'context neutral' code first time - though we still don't get that right every first time, of course.
Can you explain why TDD is better at this? I would hazard to guess that it would be worse because of Simple Design, and YAGNI making the solution specific to exactly one context. Even when we remove duplication, it is in one project not on many.
I guess it depends whether you already have the discipline to write context-neutral interfaces and code, or not. If you have been doing TDD for years, then you already will have learned good practice.
The reasoning is as follows. If the new code you intend to write has a lot of dependencies on the context, then the test harness you would write for it will need to mimic or otherwise represent all these bits of context. We naturally like to get more value from less effort (cynics call us 'lazy'), so we would think of ways that we could design the code with fewer dependencies on the context so writing the test harness would be easier. Code wih fewer dependencies on the context is more likely to work in new contexts.
That's the theory, it seems to work in the real world. YMMV.
Brian Marick takes us through a quick tour of the most important values and challenges to adopting Agile successfully (they aren't the typical challenges and values we hear in the community).
The line between development and architecture is tricky. Does it exist at all? Is an ivory tower actually needed? There's a balance in the middle, but how do you move from developer to architect?
The word 'authority' sometimes produces an allergic response in hard-line agilists. Freedom and authority – both are bad if misused and both are good if used in right spirit for a noble cause.
"Getting Started with Grails" brings you up to speed on this modern web framework. Companies as varied as LinkedIn, Wired, and Taco Bell are all using Grails. Are you ready to get started as well?
Those familiar with only ITIL V2 often scoff at the thought that ITIL could serve as a governance framework for SOA. With ITIL V3, the focus of the framework shifted towards service-orientation.
SpringSource CTO Adrian Colyer discusses AspectJ, SpringSource's dm Server and tc Server products, OSGi and Scrum.
Heroku's Adam Wiggins talks about Rails, Background Jobs, Add-Ons, Ruby, and how Heroku manages to work around Ruby's inefficiencies using Erlang and other languages.
For Grady Booch the foundation of a good architecture is patterns, SOA being just one of many patterns. In this Second Life presentation, Booch attempts to bring more clarity on what architecture is.
14 comments
Watch Thread Reply