Is iterative and incremental development à la Agile practices - where one builds only what is required per iteration - detrimental to good design? Does Scrum encourage ignoring architectural issues? Can design and architecture evolve effectively without the technical Agile practices? Does test-first development lead to good design? Or does the red-green-refactor loop stall at local-minima?
Alan Shalloway of Net Objectives asked the members of the ScrumDevelopment yahoo group about their experience building applications using Scrum if they are finding that the quality of their code holds up:
When I ask people in my courses and at conferences where is the time spent in adding new features to an existing system - writing the feature or integrating it in - typically everyone answers it is in the integration cost. I know this is because most people don't write flexible, extensible, maintainable code incorporating the concepts of design patterns in. Most of these people are also not using Scrum.
Shalloway then explained his concerns in more detail:
... they [developers on a Scrum team] build the system a piece at a time as Scrum tells us to - based on customer value. Sometimes a strong architect on the team will be able to keep things organized well. Many developers have a good sense of stepping back to see where some implementation should be added and then putting it in the right place. But many more don't know that a procedure that is implemented differently will cause redundancy if it is copied and pasted and then changed (even completely). That in this case, a template method may be a better solution.
I may be pessimistic here - but my sense is that if you don't pay attention to things entropy means they will evolve into states you don't want them to. In your case, you were using Scrum to manage the improvement of the thing I think many teams ignore. So my question is:
- do people ignore this?
- if they do, does it cause a problem?
The responses so far seem to indicate that many using Scrum have this problem even teams that write tests and refactor regularly. We would expect that teams that do not use some variant of the Agile technical practices are in even more trouble. (Remember that Scrum is development-technique agnostic - you can build your software traditionally, with XP, or any other set of technical practices as long as you do so witin the Scrum framework.)
Which brings us to the next point - how effective is the Red-Green-Refactor loop in reality? Even if a team is diligent in writing tests and refactoring mercilessly does this lead to a good design? Paul Jones blogged about how TDD creates Ravioli Code alleging that code developed with TDD creates code that is non-cohesive:
I think that a lot of TDD and test-first idealists and evangelists end up with ravioli code that is well-tested but still difficult to comprehend.
It is probable that Jones doesn't grock TDD because it is much more about design than it is about testing. But this still brings up an interesting point. How well are we in the Agile community using our practices? As the community grows are our practices causing more trouble than we expected? Are these tools more dangerous to those incorrectly using them than traditional development techniques?
Finally, let us assume that we are practicing Scrum correctly and practicing TDD correctly along with it. After all - as Ron Jeffries so eloquently showed us in We Tried Baseball and It Didn't Work - we can't blame the practice if we don't execute it correctly. Does TDD lead to good design? Refactoring is a local optimization technique when used in the Red-Green-Refactor loop. We optimize the design at the local level. This is essentially equivalent to the steepest descent algorithm. That means that refactoring will almost always lead to suboptimal designs as evidenced by the soduko solution using TDD.
Is it time to revisit our practices with a critical eye?