Opinion: Refactoring is a Necessary Waste

by Amr Elssamadisy on Dec 18, 2007 |

Refactoring is one of the key technical practices in the Agile developer's toolkit.  Refactoring also has no measurable customer value by its very definition - it involves changing the structure (design) while maintaining the behavior.  In the Lean world - anything that does not have customer value is waste, and a customer only perceives behavior/functionality and not structure. 

But there are two types of waste in Lean: pure waste, and necessary waste. Pure waste is one that has no value to either the team building the software or the customer using the software.  Necessary waste, on the other hand, is the best way we currently know how to do a job even if it does not have customer value.  Refactoring is clearly an instance of the latter. 

So, why call something out as necessary waste if it is valuable anyway?  Well, the point is, it is not valuable to the customer.  Therefore we should minimize it and be constantly looking for better ways to do what we do without it.  But, if we do not recognize it as a waste it will never be addressed because our perception of it is that it is the only way of doing things correctly.  (Think Big Upfront Design.)

If you are with me so far, then the next logical question is "so what?  How does this change things?"  By seeing Refactoring as a necessary waste, then a developer will minimize refactoring; that is only refactor code that no longer meets the requirements of the user.  This means, when you are coding away and you see a method in the class you are modifying that "smell's bad" but has no direct connection to the requirement you are working on, you leave it alone. 

Related Editorial

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Add some slack! by Torreborre Eric

I wholly agree with lean principles but programmers are not machines! Adopting this precept with a hard line can also have drawbacks.

Every now and then you leave methods which are not totally properly named or code which is slightly not readable. Because you don't see immediate profit for the customer in doing so. Then you may just become less productive because it's not so nice anymore to work on "average code doing the job". In a way, it could be seen as a local optimization at the detriment of the whole picture.

To that respect, I like Tom de Marco's book: Slack. Slack isn't necessarily waste,...

Eric.

Perhaps.. by Zubair Hamed

Its true to a point. Refactoring for the sake of refactoring is perhaps a waste, however if refactoring translates to easier maintainability of code and easier readability perhaps (software systems often outlasts the developers), i think its a necessary 'evil'.

Z

Re: Perhaps.. by Zubair Hamed

I forget to say that these reasons may translate to business value in the form of $$ if it, say, reduces man hours in maintenance.

Z

Code that meets the requirements of the user by Jeff Santini

If you are changing code to enhance it, then you, as the developer are a user of the code. If you can do your job easier now and the next time the code needs changing then one could argue that improving the code is not waste, while it continues to meet the requirements of the customer. For instance a variable named x could continue to meet the customer's needs, but as a developer if you rename it to currentBalance you add value to the code base. I don't believe there is anyway to call that waste, assuming the variable does refer to current balance.

I do agree it could appear to be waste to a customer, but it does add value to the code base, again assuming it helps the developer complete his job and/or makes the code more meaningful to those who need to understand it.

It's paying back technology debt by Lari Hotari

Here's one way of thinking about refactoring. Without refactoring a solution, you take technology debt. The debt grows interest with an unpredictable rate. The later you pay back, the larger the debt will get. When the debt gets large enough, it will be impossible to invest in new features. It's only recommended to take short-term technology debt and pay back early.
Here's one blog posting about technology debt vs. financial debt: onstartups.com/home/tabid/3339/bid/165/Developm...

By analogy by Michael Nygard

I see some comments with a defensive tone about refactoring, but Amr is not suggesting we should do away with it.

In a factory-floor implementation of lean, the equivalent to refactoring would be things like preventative maintenance and cleaning the shop floor. Being Lean has never required sloppy work practices or poorly maintained equipment. Just the reverse! Badly maintained machines create defects or, worse, injuries.

In fact, part of the discipline of eliminating waste is to find preventative measures that will keep the flow moving. In our world, refactoring is that practice.

Now, what if a new machine comes along that requires 50% less maintenance but can keep up the same level of safe, high-quality production? Naturally, you would want to switch over to the new machine. It would help you reduce your necessary waste.

Similarly, if a new language, programming technique, or toolset came along that would help you maintain high-quality code production at the same velocity, but with 50% less refactoring, you would want to switch to that technology.

If I had an axe to grind, this is where I would shoehorn my pet technology into this argument. Instead, I'll just say that, right now, refactoring is the best technique I know for maintaining production flow.

Can You Taste The Waste? by Kevin Teague

Ween summed up this article in their song "Can You Taste The Waste?"

Re: It's paying back technology debt by Amr Elssamadisy

You are right Lari about financial debt. But, there is not one piece of financial debt for an entire application, on the contrary, there are several of them for each element of design.

Now, say that there is one piece that is really quite bad. At the same time, that piece is not affected by the current requirement at hand. Or the next. Or the next. It is infrequently visited.

When should you pay off that debt and refactor it?

What I am suggesting here is that you not pay it back until you have to because refactoring has no customer value.

Re: It's paying back technology debt by Bruce Rennie

That's very true, but it's not really the point.

The goal is to produce customer value. Technical debt gets in the way. Therefore, technical debt should be reduced or eliminated. The current, best practice to achieve this is refactoring. It's not that refactoring is great, it's that technical debt is really bad.

If we were omniscient, we'd design on the fly with no technical debt. If that were the case, would we even have a concept called refactoring? Of course not. If we completed a project that achieved all the customer goals and never had to do a minute of refactoring, would that be bad? On the contrary, it would be great.


Refactoring is a reaction/solution to a problem. That makes it necessary waste at best.

The value of 'customer value' by Amr Elssamadisy

In the lean world, the only value is value to the end user. There is no intermediate value, value-add, or any such thing.

In a doctor's office, there is no value in sitting at a doctor's office waiting for your turn in line. I don't know about you, but if all things were equal, and I found a doctor where I didn't have to wait, I'd switch!

Also, think of a bicycle factory that has multiple steps. One of those steps is 'allow the paint to dry'. To me, the customer, that step also has zero value. Sure, I don't want a bicycle that gets paint on me, but that's beside the point. The wait time for the paint to dry is waste - but a necessary waste.

So why call something 'waste'? Is it just to be intimidating? No. Without calling something out as waste we will miss many opportunities to improve our situation - the problem that isn't there can't be fixed.

Now, back to refactoring. Refactoring helps us to do our job better. It is currently the best way I know to build software. But it has no customer value. So what?

Well, two things: 1)Knowing that it is waste, we will be careful in its use. We will only use it when necessary and not gratuitously. 2)We will be able to recognize better solutions when they come along. When the next advance in technology comes along we will see it.

Re: Necessary waste? by Jeff Santini

My discomfort is with the idea of necessary waste. It is either necessary or waste. Unless you start redefining these words, they can't both be used to describe the same thing.

Are we really just trying to say there is good refactoring and there is bad refactoring?

Re: The value of 'customer value' by Jeff Santini

But the waste does not come from refactoring, the waste comes from the activity that necessitated the refactoring. Believe it or not I don't want to ride a bicycle with wet paint:) So once my bike has wet paint I want to wait and be told that I need to wait. Now if you could avoid wetting the bike that would be great. But I would say the step to let it dry is not the step that creates waste. The step that wets it is the one that creates waste.

I hope it does not sound like too much hair splitting, but fixing a problem is never waste. Creating the problem is waste. Any attempt at refactoring which results in less valuable code was a failure. Refactoring by definition improves the code. It was the step that necessitates the refactoring where the waste is to be found.

Re: Necessary waste? by Bruce Rennie

I would submit that necessary waste is one of the most important concepts in lean.

People can usually identify value. They can usually identify waste. It's those things that we do with good intentions that don't directly contribute to customer value that can get us into trouble.

The idea of necessary waste tells us that there are, or should be, limits to how far we pursue a particular activity. Refactoring may be valuable, but I can sink a company by re-writing my code daily. I can't do without refactoring, but I don't WANT to do refactoring. Necessary, but wasteful.

Re: The value of 'customer value' by Bruce Rennie

It's not about "does refactoring create waste?". It's about "does refactoring create customer value?". By definition, it does not. Therefore, it is waste.

I would also disagree that fixing a problem is never waste. Identifying waste is always a good thing. Acting to eliminate it is ultimately a business decision. If the business fails or suffers as a result of pursuing the elimination of waste too aggressively, then was that the right decision?

Re: It's paying back technology debt by Pete Johnson

Here's one way of thinking about refactoring. Without refactoring a solution, you take technology debt. The debt grows interest with an unpredictable rate. The later you pay back, the larger the debt will get. When the debt gets large enough, it will be impossible to invest in new features. It's only recommended to take short-term technology debt and pay back early.
Here's one blog posting about technology debt vs. financial debt: onstartups.com/home/tabid/3339/bid/165/Developm...


I couldn't agree more with this take on refactoring or software architecture in general. The marketing folks never want to pay the price to make things more easily scalable in the future and it absolutely is a debt. I like the language used to describe it that way very much, thanks Lari.

Pete Johnson
HP.com Chief Architect
Personal blog: nerdguru.net

Re: The value of 'customer value' by Javid Jamae

Waste *absolutely* can come from refactoring. Waste is anything that doesn't produce customer value. Think of technical debt as potential energy and refactoring as kinetic energy. In fact, I would argue that there is more waste in the refactoring than in creating the technical debt. When you are creating the technical debt, at least you are producing something that is of value to the customer.

Don't get me wrong, refactoring is a necessity based on our current model of software development. It is our professional responsibility to deliver clean code, and refactoring is currently the best way to achieve that. But... if debt exists in a part of the system that is not touched often (or ever) then there is no value in refactoring it. If you do, you are taking time away from refactoring more used parts of the system, or from adding new functionality. The point is that a system can have evil, but if that evil is contained, functional, and seldom updated, there is little value in making it good.

All this being said, if we want to look at things from a Lean (and Theory of Contraints) perspective, refactoring is not the most significant waste to try and minimize for most organizations. In fact, I've never been in an organization where we could achieve a big bang in productivity or delivering customer value by being mindful of what we were refactoring. Refactoring is something that you do within a 10-30 minute timeframe. Most of the time, developers know naturally when they are veering too far away from the story they should be working on and they get back on track.

So, while I agree with Amr in principle, I think there are more important things to worry about. :)

Re: Necessary waste? by Jeff Santini

Well I don't want to drag this rather detailed discussion of semantics too far, I still believe that a necessary activity and a wasteful activity are mutually exclusive based on dictionary definitions of the words necessary and wasteful, but can accept the potential benefit of a Lean phrase that helps make one think about the difference between the two. But I also must say that Refactoring, again based on accepted definitions, is never necessary, it can only rise to the level of desirable, though it can drop below that threshold as well.

Re: The value of 'customer value' by Jeff Santini

Agreed, fixing a problem could take more effort than the effort required to live with the problem. So it could result in waste.

But I liken refactoring to keeping your work space in order. Certainly you could spend too much time keeping your work space in order, but in the bigger scope of things that is probably not the first place to look for waste.

Refactoring can have customer value by Peter Minearo

Refactoring is not unneccessary waste. And it absolutely can have customer value. It gives the customer value in an indirect way. One way it by reducing time to market. Technology debt is one of the biggest factors in time to market, right along with complexity of functionality. But then the more complex the functionality and the more technology debt you have; the longer it will take to release to the customer. Refactoring reduces technology debt, which decreass time to market. If I can get a piece of functionality out to a customer faster; they will see that as having value. If you don't believe me; take a look at eBay. They release out to the US and Canada every other week. Why? Because their customer's are demanding functionality and want it as quickly as possible. That is the value to the customer.

What this all boils down to is this: "Software Development NEVER gets it right the first time." We always have to come back and make changes. That is why it is better to build applications in small increments; instead of big upfront designs that take a lot longer to accomplish. The market can change by the time you finish the project, which results in a big waste of time for the company. The biggest factor is that companies DO NOT know what the customer wants all the time. So, they need to be "agile" and have the ability to release functionality very quickly (and fix bugs). Which is next to impossible when the application has a large technology debt.

Refactoring Captures New Knowledge by tom poppendieck

In product development, as contrasted to manufacturing, one of the biggest wastes is lost knowledge. Refactoring is an tool to capture learning and improved understanding in the code.

When we adopted lean and total quality some decades ago at Honeywell, we learned that there were two categories of work worth doing, one was things of direct value to the end customer; the other was things that improve our capability or capacity to deliver value to the customer. Things like refactoring and process improvement fall into this second category. One difference is time frame. Capability and capacity improvement tends to have a longer term payoff, perhaps over several products or projects. Yes, there is always a trade off between investing for the long term and delivering immediate value and this trade off must be made continuously by every member of the whole product team (not just programmers).

Most of the effort in creating software goes into learning, not coding. Much of what an agile team learns is expressed in its code base and tests. Refactoring is a tool to express a new understanding of some aspect of the problem the team is addressing.

Refactoring is not the problem, the real question is what areas to invest time in to understand better. Choosing to invest in things that will not matter either to either the external customer or to the organization either in the long term or the short term would be waste. Having made the decision, large or small, to invest time in learning about something, the waste would be not capturing the knowledge. Bad decisions about what to invest in are an instance of what Alan Ward ('Lean Product and Process Development' 2007) might assign to the category of waste he calls scatter. One might label this 'drive-by refactoring'.

Re: Refactoring can have customer value by Dave Rooney

I believe this is a case where the application of the manufacturing metaphor of Lean falls down. In manufacturing, the design is done by the time we get to the point in the process where we want to eliminate waste. In the case of software development, we're constantly designing and redesigning the software.

As Tom Poppendieck has already mentioned in another comment, we learn more about the code and design as we go. The principles of Refactoring allow us to codify what we have learned in order to improve existing functionality or incorporate new work. Describing this as waste, even necessary waste, is trying to fit the square peg of the metaphor into the round hole of reality.

Regarding "Software NEVER gets it right the first time", well this is the Agile section of INFOQ, and Agile does work from the premise that we know that we aren't going to get it completely right. So, refactoring is absolutely necessary in that case and does provide tangible business value through reduced technical debt.

Dave Rooney
Mayford Technologies

An analogy with aerodynamics by Dean Schulze

I normally don't like analogies, but I think this one works well.

In aerodynamics there are two types of drag. Induced drag is inherent to creating lift. Parasite drag is due to the frontal area of your airplane and the friction of it moving through the air. Parasitic drag has no benefit, but induced drag is necessary to fly.

Refactoring due to poor initial design or code is analogous to parasitic drag. Refactoring to support new features is analogous to induced drag. You want the former as close to zero as possible, while the latter is the price of adding new functionality. Calling the latter kind of refactoring waste is like saying you're wasting power when you fly due to induced drag. You have no choice if you want to fly.

A situation that occurs even in well designed and coded projects is that your patterns and coding idioms will sometimes evolve over the life of the project. When that happens refactoring old code (that was well designed and well written at the time) to bring it up to current standards is a judgement call. It's hard to tell if this kind of refactoring will pay for itself through reduced maintanence costs.

Re: An analogy with aerodynamics by Frank Bolander

I think you're using refactoring in too broad of a context. I will agree that in normal SDLC iterations refactoring is a parasitic drag. However, when business conditions warrant a redesign because of need or desire to capture new business(ie induced drag), that is rearchitecture, not a refactor. It may be semantics, but I think the terms refactor and rearchitect are orthogonal concepts. A rearchitecture usually has a charter attached to it with hopefully tangible risks/rewards. Refactoring is just an ends to a means to meet customers expectations during a current project scope.

Refactoring is a problem from a standpoint is that it is hazy at best when trying to scope out a project and more importantly, a budget. If a customer goal is met, why would he/she allocate any more funds to refactor?
Speed to market capture is more important nowadays than "code beauty".

Quantifying the value of refactoring by Kent Beck

While it's true that refactoring doesn't deliver features to customers, features aren't the only source of value in software. Another source of value is the option value of the software--software that could be made to do any one of five things next is more valuable than software that can only be made to do two things next. Refactoring can add to the option value of software.

This perspective can also help teams select the most valuable refactorings out of the seemingly infinite pool of possibilities. Perform refactorings that add to the set of available and useful options, defer those that don't.

Re: The value of 'customer value' by Kent Beck

I gave a course once where I described incremental design in great detail. Towards the end one of the participants put up his hand and plaintively said, "Wouldn't it be easier just to get it right the first time?"


My reflexive answer is "yes, but I don't know how". On further reflection, I think the answer depends. If I compare a hypothetical design process that results in a system that doesn't need refactoring with an incremental design process that quickly delivers real usage data, the incremental process can easily be more valuable. If the feedback from use is not valuable or if the cost of the increments of design are high, up-front design would be more valuable.

Kent Beck
Three Rivers Institute

Re: Refactoring can have customer value by Javid Jamae

I think the point here is that refactoring it is "necessary waste" not "unnecessary waste".

Waste is an overloaded term. When someone says "that was a waste" they typically mean it was not at all worth doing. In Lean when you say that something is a "necessary waste" that means that it *is* worth doing and it must be done, but that the act itself doesn't provide any direct value to the customer.

Re: An analogy with aerodynamics by Dean Schulze

Frank,

Where does redesign end and re-architecture begin? I know re-architecture when I see it, but it's hard to define.

The customers' goals are continually changing, which is the only reason they pay for additional features and the refactoring to support them.

Re: The value of 'customer value' by Amr Elssamadisy

We are in hypothetical land now, but.... If we could design something right the first time - I'm not so sure that it precludes delivering it in incremental steps :)

Re: Quantifying the value of refactoring by Amr Elssamadisy


Perform refactorings that add to the set of available and useful options, defer those that don't.


<tongueInCheek>
Hmmm.... I thought simplicity and deferring commitment kept our options open and not refactoring.
<toungeInCheek></toungeInCheek>

Thank you Kent for your valuable input!

</tongueincheek>

Re: By analogy by Amr Elssamadisy


I see some comments with a defensive tone about refactoring, but Amr is not suggesting we should do away with it.


Thank you! I am just suggesting that we be aware that it does not produce customer value - to keep our eyes open for a better way of doing things.

Re: The value of 'customer value' by Amr Elssamadisy


All this being said, if we want to look at things from a Lean (and Theory of Contraints) perspective, refactoring is not the most significant waste to try and minimize for most organizations.


Let's make this interesting and pull in the ToC idea of only addressing the bottleneck:

Would you agree Javid, if unnecessary refactoring was being done in a bottleneck step then it should be removed and that it will have an impact?

Re: Refactoring Captures New Knowledge by Amr Elssamadisy


Most of the effort in creating software goes into learning, not coding. Much of what an agile team learns is expressed in its code base and tests. Refactoring is a tool to express a new understanding of some aspect of the problem the team is addressing.


Yes. But ONLY when Refactoring is pulled by a user requirement. If, on the other hand (and many of us are guilty of this), if we are refactoring because we just happen to see a method that looks ugly, but is not related to the new learning/ new requirement at hand, then Refactoring does not even help our learning.

Re: Quantifying the value of refactoring by Amr Elssamadisy

sorry... one too many cut&paste - the Thank you is genuine.

Re: Refactoring Captures New Knowledge by Deborah Hartmann

there were two categories of work worth doing, one was things of direct value to the end customer; the other was things that improve our capability or capacity to deliver value to the customer. ... One difference is time frame.


Quite so. I think we can oversimplify "value to the customer," using it to manipulate the customer to our convenience. I.e. we estimated badly, so we cut corners, stop thinking, do a lot of cut-and-paste, whatever. Oh, and we don't tell them that it will come back to bite them later, and they can't tell because it's not "visible" in the demos we do with them.

Depending on the situation this cheats either the long-term customer who owns the code or the team that hopes to continue delivering at the same velocity using that codebase.

Revealing these issues to the internal or external customer is a responsibility of the technical team, as part of collaborating toward a common goal. When this is done, the customer can participate in the choice to spend on refactoring. At very least, refactoring isn't "invisible" in terms of cost!

Not a necessary Waste but a hidden value to the customer by Kiran Rao

Refactor maybe considered as necessary waste in Lean, but in my perception we are adding some value to the customer also. You may ask how..?
The code base that is going to the customer, will be their intellectual property. If we deliver a code that is clean and adaptable to future change it is actually a value to the customer in the long run, though it may not be considered a value at present.

Re: An analogy with aerodynamics by Dave Rooney

Aviation analogies - right up my alley! :)

Consider the venerable Boeing 737, the most produced commercial passenger aircraft in history. The original -100 and -200 versions were substantially different than the newer versions. One important difference is that the older versions had JT8D engines, which have a much smaller circular cross-section than the CFM-56 engines that are used in all of the newer variants. As a result, the parasitic drag of the engines and their nacelles is higher. However, the CFM-56 is more efficient in terms of fuel burn by about 30% so the increased parasitic drag is more than made up. Additionally, the CFM-56 engines require only about 50% of the maintenance of JT8D's, and are therefore even more efficient from a cost perspective.

Older code can be very much like the JT8D's. It works, and has delivered value for a long time. However, a newer more efficient approach could be used that will incur some cost for the "re-engine" exercise, but there will be significant long-term savings. Think back 7-8 years... in the Java world, we were using EJB's or hand-rolled JDBC code (or both). Enter tools such as Spring and Hibernate (and many others), and we discover that there are simpler and more easily testable ways to build systems. Do you just ignore new approaches, or do you "replace the engines"? It's not a simple question and the answer will be very dependent on the context.

One cautionary note, though. I recently talked to some developers who had to dust off the code for a system that was supposed to be mothballed, but it's replacement system had been, um, "paused". They were using Sybase PowerJ and EAServer 3.6.1... right out of the summer of 2001! They want help in moving as quickly as possible to new tools such as Eclipse. They will probably spend several months just converting to new tools let alone delivering any business value. Perhaps they should have spent a little money at intervals over the past 6 years doing that.

Dave Rooney
Mayford Technologies

Re: Refactoring Captures New Knowledge by Dion Stewart


Most of the effort in creating software goes into learning, not coding. Much of what an agile team learns is expressed in its code base and tests. Refactoring is a tool to express a new understanding of some aspect of the problem the team is addressing.


Yes. But ONLY when Refactoring is pulled by a user requirement. If, on the other hand (and many of us are guilty of this), if we are refactoring because we just happen to see a method that looks ugly, but is not related to the new learning/ new requirement at hand, then Refactoring does not even help our learning.


Let's say you're implementing a new feature and while you're in the code you find several places where a large block of code is duplicated. Let's also assume that this duplicate code has nothing to do with the new feature being implemented.

By your definitions, removing the duplicated code through refactoring provides no value to the customer and it does not contribute to any learning about the software.

Are you sure about those two things?

Removing duplicate code provides no value to the customer? Wouldn't refactoring the code so there's no duplication actually be eliminating waste?

Finding duplicate code doesn't foster learning because it's not related to the new feature? Really? Seeing classes/methods that share behavior that you were not aware of before isn't learning?

Re: Refactoring can have customer value by Stephen Cresswell

Well said! I'd like to add that refactoring regularly enables repeatable delivery on top of reducing time market. Depending on the circumstances this can also be valuable to the customer. There are occasions when teams spend so much time refactoring that they aren't delivering value quickly enough, but this should be seen as a symptom of a different problem.

- Were the requirements understood before development started?
- Are the team using frameworks they don't understand?
- Are the team good enough developers?
- Were their sufficient quality checks in place (e.g. pair programming)?

Fixing the above will reduce refactoring and improve code quality, leading to fast repeatable delivery. Not refactoring / ignoring the symptoms of such problems will lead to code rot, higher defect rate (hugely wasteful), slower time to market and missed delivery dates.

Re: Code that meets the requirements of the user by Martin Schapendonk

Unfortunately, most developers aren't sponsors of their own work, so the concept of 'business value' (which is pretty vague, but that's another discussion) doesn't apply to their own 'feelings' about the code base.

Does throughput have customer value? by Dries Bellen

Sometimes you just have to step back to jump further. If refactoring some code enables you to speed up development, it can mean that your time-to-market is shorter, which definitely has customer value.

The only question in this matter is whether you do this proactively or on demand.

Re: The value of 'customer value' by Javid Jamae

Amr said:

Would you agree Javid, if unnecessary refactoring was being done in a bottleneck step then it should be removed and that it will have an impact?


Absolutely! Martin Fowler (author of Refactoring) even says that refactoring code that won't likely change is of no value:

"One misuse is that of refactoring code that won't change.The whole point of refactoring is that it improves the design of existing code. The value of well designed code is that it is easier to change. Hence you refactor code that you expect to change in the future. There's no point refactoring code that's stable.

-- martinfowler.com/bliki/RefactoringCringely.html

Re: Refactoring Captures New Knowledge by Javid Jamae

Removing duplicate code provides no value to the customer? Wouldn't refactoring the code so there's no duplication actually be eliminating waste?


Yes, refactoring does eliminates the waste created by surplus code, this is why it is necessary. But it also adds waste because it takes away time that could be spent adding direct value by adding features. Hence it is "necessary waste".

Does removing duplicate code provide customer value? It depends. If the code that has the smell is not part of the feature you are working on and is stable (is not likely to change), then no, it provides absolutely no value to the customer. We have to accept that we have long term debt in the system that it is contained and stable. For example, is there any value in refactoring code that hasn't been touched in 2 years and which likely won't be updated with new features? Probably not.

If the code with the smell is not the feature you're working on but is unstable (it is likely to change), then there is just as much value as refactoring the code you are currently in. Refactoring is always time spent to create potential future value for the customer, but that potential value is only realized as real value if the refactoring actually makes the development of a future feature go faster. This is a risk, but it is a risk that I usually take.

Summary and Thanks for the discussion! by Amr Elssamadisy

First of all, I want to thank everyone for sharing their opinions. I've enjoyed the conversation, learned a lot, and have come away with a few things to think about. Now that the flurry of posts has come down, I wanted to summarize a bit of what I've read so far:

1) Not all refactoring has value to the customer, some has indirect value and some does not. It is proportional to how much the code that is refactored is used.

2) There is value to the customer that is not directly seen and should probably be communicated. This includes, but is not limited to:

a)faster throughput - which reduces time to market - via reduced technical debt,
b)it reduces another waste - lost knowledge - by encoding the knew knowledge learned by iterating
c)it increases the options available to the customer by making it more simple to add new features later on

3) Taking those who left comments as a sample of agilists in the field, it seems that we are not all crystal clear on the benefits of refactoring to our customers. And, more importantly, when it should not be done.

Amr

Re: Summary and Thanks for the discussion! by Deborah Hartmann

Thanks Amr, for the summary.

Of course, the conversation can still continue! :-)

Re: Summary and Thanks for the discussion! by Javid Jamae

Not all refactoring has value to the customer, some has indirect value and some does not. It is proportional to how much the code that is refactored is used.


No, not how much it is used, the value of refactoring is proportional to how much the code is updated. It is the stability that is key. You could have stable code that is accessed thousands of times a second in a production environment. There is no value in updating this code and in fact you take on more risk if you do. (Risk is function of likelihood and consequence, the risk of breaking more frequently used code is greater because the consequence of breaking the code is greater.)

That brings a whole other dimension into the picture that we haven't discussed. Studies have shown that a lot of features go unused in software projects (45% I've heard). I've argued that teams should put more emphasis on removing unused features to reduce complexity. If you spend time refactoring code that is largely unused (even though the product owner asked for it) you are also creating waste.

It would be interesting for an IDE to provide some type of visual indicator - a heat map of sorts - to signify what parts of the code have changed the most over a 2-3 month moving average. That could somehow be overlaid with data on actual production usage of the code. That would give you a good indicator on whether you should pursue the refactoring or defer it. I'm guessing there are already analytical tools available to derive the frequency of change information from a source repository. You would have to build in some mechanism for evaluating actual feature / code usage.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

45 comments

Educational Content

General Feedback
Bugs
Advertising
Editorial
 InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy