The Decision to Refactor
Refactoring is the process of changing a software system in such a way that is does not alter the external behavior of the code yet improves its internal structure. The idea of improving an already written code is appreciated in most Agile teams. After all, continuous improvement is is something that these teams strive for. However, improving the already existing code involves time and money. Is it worth it?
Refactoring involves costs, these are
- Cost of Doing the Refactoring
- Cost of Testing the Change
- Cost of Updating Tests and Documentation
It also has the risk of introducing new bugs into the system if the Unit tests and the test harness is not upto the mark. On an XP project, this risk is low assuming that there are good unit tests. On the flip side, there are several advantages offered by refactoring,
- Adding new features does not corrupt the system structure
- Improves understanding of the system
- Easier to test
- Easier to find, isolate and fix bugs.
What are the parameters that should affect the refactoring decision?
You shouldn't have to do a complex business analysis, keep it simple. How much time will it take to refactor and multiply that by an hourly or daily cost factor.
$60000 Salary x 1.25 = $75000 burdened cost
$75000 cost / 12 months / 22 working days = $297.61 (Cost per day)
Now, 3 days to refactor the code x $297 = $891 refactoring cost (This is your investment)
On the next code change, It takes 5 days to work the code WITHOUT the initial refactor (5 x $297 = $1485) If takes 1 day to work the same code that WAS refactored (1 x $297 = $297)
$1485 - $297 = $1188 (Savings) - $891 (Initial investment) = $297.00 overall savings
At this point, the investment has paid for itself, and any future savings are gravy and can be used towards productivity on other products, projects, documentation, billable work, etc.
Doing a similar quantitative analysis Simon Johnson suggested there has to be a certain percentage increase in productivity post refactoring for refactoring to justify the cost. According to Simon,
For refactoring to be a better investment than just banking the cash, you have to improve productivity by 5-8% or more. This is quite a big ask and I have big doubts this can be achieved in practice. If your opportunity cost is much higher than the figures represented by "Account B" then you're probably best spending the money on something else, like new features or defect reduction techniques.
However, apart from cost, there could be other reasons due to which refactoring might not be a viable strategy.
Mark Needham commented on the refactoring dilemma citing the case of projects where the deadlines are tight. Team would not see any returns on the refactoring effort if they started at a given point. Mark suggested that though there are several benefits of refactoring however, he has seldom seen delivery dates being pushed out just because refactoring would make it easy to maintain code in the future.
Likewise John too presented scenarios where refactoring might not be an option,
Generally speaking though, management may also have a very good reason not to refactor. It might have nothing to do with money or time, it could be a strategic decision or even a contractual reason you are not aware of (i.e. government work). Make sure you know all the whole story too. I also see a lot of people talking about doing it on your time. This certainly shows conviction and love of your craft, but I wouldn't do it in a vacuum. I have seen programmers do this and cause havoc for other team members and the project as a whole.
Mark also suggested that sometimes the lifespan of the project might be too short-lived to invest in refactoring. Investing in refactoring for such a project might be a futile excercise.
Hence, though there are definite benefits involved in refactoring, the decision to refactor or not, at a given time, in a given context has to be taken with caution. The key lies in weighing the benefits and then taking a call which adds the highest business value.
Re: Technical Debt
Not doing it refactoring, thus not caring about your code base, has a much higher costs than can be anticipated, not having complete test harnesses up, with automated test that does not require changes when doing refactoring is just plain professional practice, how else will you keep the pace up and the bug reports down. As soon as you stop doing all these good practices you suddenly see that getting a release out of the house suddenly doesn't take hours it goes into days than into weeks and even worse maybe even months of "stabilisation" as it used to be called.
Suddenly your workshop floor has no clean surfaces, no shelves to offload stuff on, no gas tubes for the welds and no garage doors that open. And all this happens in weeks, not months or years, it is just so gradual that people forget what things used to be like when the code base was nimble and agile. It is a lot like aging, you don't see it yourself in the mirror, but everyone else does.
Re: Technical Debt
Also management should not have any other agenda, they should provide their developer with the entire view, all reasons, requirements and strategies, otherwise they cannot make informed decisions on when to improve certain areas in the code and when to leave for a rainy day.
Other benefits of refactoring
When dealing with a unknown codebase refactoring can help you to understand the code, its intent and structure. So you refactor to learn about it. It depends on other factors if you will keep the refactored code afterwards.
Refactoring is small steps. But it often leads to a point where a insight about the design pops up. Then a big laps can bring the design forward quite a lot. I often compare that to one of the last puzzle pieces falling into its place and giving a very different view than just before.
And refactoring is an ongoing continuous process. If you keep your technical debt low and follow the boy scout rule, it is not much effort per day (certainly not more than surfing the web during work hours) to steadily improve/keep your code quality.
Don´t even ask
Even starting to justify refactoring is plain wrong. Refactoring need not be brought to the attention of any non-technical management like a Product Owner or a project leader or the customer or the company´s boss.
Refactoring needs to be just included in any estimates. No explanation needed for "why does that take sooo long?"
Never ask permission for doing your job in a professional way. That means: never ask permission for refactoring. If you just refactor as needed from the start on, then there won´t be huge refactorings in the future.
Re: Technical Debt
In fact, the features added without refactoring will likely themselves create additional clutter and technical debt, simply because there is no good place to add them cleanly. So this truly becomes a compound interest problem, where cost starts to rise exponentially.
And don't forget the cost of demoralized developers.
Tom Gilb & Kai Gilb Jan 26, 2015