Confessions of A New Agile Developer

I have worked in Waterfall model for most of my career. Some time back I joined Xebia and started working in the Agile style. Specifically, we have been following Scrum and XP methodologies with TDD as an emphasized practice. The transition from Waterfall to Agile is like walking from one planet of the universe to some other planet. When you shift from one to the other there is a complete change in your world, everything from the way you think, work or collaborate, changes.

I joined a team of 8 professionals and contrary to my expectation 6 of us had no prior experience with the Agile approach. So we started under the guidance of our 2 experienced pros. Initially there were few ups and mostly downs. We found out Agile is not just about sprints and no documentation - there are more subtle aspects which you have to learn the hard way.

Here is an account of what went on:

Some Startup Hiccups:

1. The Thinking Shift

Agile is all about thinking in the present, the current sprint, the current user story etc. We do not think about how things will work when some other story that we might encounter down the line comes to the board. In Waterfall we used to think about the whole system first, the HLDs and LLDs were the first steps and they had to be frozen before moving forward.

In contrast, Agile is all about current state and constant change, if our requirements change in the future, our design will evolve accordingly; but we do not concentrate much beyond the current sprint. It takes some time to absorb this fact.

2. Changing Gears

Agile is a constantly changing environment. “Responding to Change” is one of its prime directives. In order to respond a developer must be in sync with market technologies else it will cost him a lot of pain.

We were using a search engine library in our project but only one of us had knowledge about it. Due to this handicap we had issues while estimating things, while pairing up, during stand-up meetings, and other day-to-day practices.

The other handicap that we faced was doing TDD appropriately. From a technology perspective, TDD has quite a few options. You can rely on various mocking and testing frameworks, which you do not use that often in Waterfall. A comprehensive knowledge of these is a must if you follow TDD else there will be a lot of hiccups.

The Sprint Slowdown

1. Test driven development is all about writing tests first and then deriving business logic from the tests. This is one of the hardest things to adapt to, as it involves a complete change in your perspective.

In Waterfall there is code first and then some tests (usually, but not always); but TDD it is all about Red, Green and Re-factor. The approach asks us to think in abstract terms and then evolve concrete things from them. Since it is hard to adapt at first, we often resorted to the practice of writing the logic first and then making sure that there are tests to cover them. I would call this as Development Driven Tests (DDT) approach.

DDT does not add much sense. Why make a test if you know your code will work fine? Just to prove your point that the code works? There should be more to it than that. And there is. For example, writing tests does make your code better in terms of coupling and provides a huge safety net when making changes in the future.

2. Thinking in abstract terms is a must for developers in Agile. In Agile we do not think of an upfront design, as in Waterfall, the design emerges in due time by creating abstractions and developing work flows. Developers in Agile must be well versed with at least the basics of deign patterns else they will litter the code a lot and only crap can emerge out of it.

TDD assists us a lot here. It asks us to think in abstract terms. Moreover as soon as we are done with fixing our tests we must think about and refactorings that can make the code better. DDT also helps identify right away, any quality issues so they can be addressed before it is too late. So we eventually learned that we must throw away the code first – test second approach, and then we re-started in the TDD manner.

3. The code quality is team's responsibility and people in team will re-factor or possibly rewrite it time and time again, until it matches the standards prescribed. We must not attach any emotions to our code and should be prepared to toss or rewrite it again and again. Following advice from Venkat Subramaniam, on the practice of collective ownership: “we should strive to improve the code quality every time we performed a check-in.”

4. In Waterfall requirements are signed, sort of, in blood and then you move forward but in Agile, the requirements evolve simultaneously with the solution. This helps to make things more clear as we go ahead, but they also result in demands for redesigns of the system at times and this will happen quite often in the initial sprints (1-4). Moreover the backlog can also change in the middle of the sprint, so the team should be geared to take up those changes, as delivering working software at the end of each sprint is the Agile motto.

5. Pairing initially seems to be a waste of time and effort. Two people working together on the same story looks like a waste of 50 percent of both, time and effort. Moreover multiple people working on different tasks of the same story seems to be a cause of slowdown in velocity.

But teams in Agile are meant to work like partnerships and not like individuals working on the same project or in the same room. When teams work together they push things to their best and consequently you have a greater likelihood of finishing things consistently rather eventually failing on multiple fronts.

6. Pairing is best done while following TDD. The driver and the navigator co-ordinate their efforts to develop a better system. But pairing while coding if you are following DDT approach is not the best way to go forward. In such a case neither of the partners will have any idea how to go forward e.g. what the system design will look like, so you will have lot of noise and then what emerges is a bunch of source files that must be rewritten.

The best approach is to follow TDD but if you cannot adopt to it you can still pair using a glass (white) board to first chalk out some design flow and then either of the partners can write the code while the other one can work on test cases.

7. Delivering working software in the demo is the litmus test of the sprint. But, do you consider a sprint successful just because the software is build-able workable if it is not releasable?

Is a sprint successful when there are few stories completely done with some left out, or a sprint with all of the stories mostly done but with niggling issues?

Teams should focus on getting a full story done with all the issues sorted out and meeting the definition of done rather rushing over the backlog for finishing things. A poorly implemented story is nothing but a rework that must be done going forward. Often time is a constraint, so depending on the relative importance of stories teams should look for a qualitative implementation of the solution they propose. Stories on the top of the backlog must be developed with the best solution possible and as we move down the backlog there can be some compromise over quality of the solution but not on the definition of done. However it is better to leave a story rather to implement it quite poorly.

8. Stand ups are sacrosanct. They are meant to be a shared platform, and not just your solo opportunity to tell others what you did and what's next to follow. People should try to listen to what's happening around rather checking their checklist for accomplished activities.

Sanctity of stand ups must also be maintained in terms of time-lines. We often are unable to resist the temptation of starting a discussion given a point but that must be avoided.

9. Agile teams are quite small and in such a situation people can often be judged on the basis of their stand-up updates. So should you say we were banging brains against each other or I did nothing yesterday or we re-factored the code and now we do not know what is what ? These would create an awkward situation with the new ranger.

10. Planning meetings are bound to be brain-whacking and exhausting. Sitting in the chair for 4 long hrs and deciding on the story points feels like playing roulette. The numbers will decide what will go in the sprint and what will not but how to decide the number when you had no experience at all? You are bound to lose.

But these numbers are meant to be ballpark figures which can go wrong, that does not mean you start playing roulette for estimations . The numbers will give you some order in few sprints down the line, giving an idea about the total amount of work that can be picked up.

11. Assigning story points is a complex task and should be done in phases. Teams internally should first analyze the stories and have discussions with the product owner to get a clear view of what is required. After having a clear view, a broad level technical discussion should be done considering the best possible solution that can be implemented. Failure to do so will result in ambiguity in terms of how the story should be implemented and thus a flawed estimation. Let us say that if a story touches a new unexplored domain then it should to be quite complex, even if it is simple task. Even in the known domain depending on the technical solution the story can be quite complex.

Stories can be estimated best when they are crisp i.e. a clear view of what is required with some details of how should be done. The product owner cannot supply crisp stories. They can only be made by having discussions with the team. So teams must spend some amount of time in order to gear-up for the next sprint.

12. Retrospectives look like speeches given at the Poll time. We should have done this or that, and in the next sprint we can do this or that, but nothing happens if some tasks are not taken in the sprint. People can express their comfort or discomfort with different aspects but teams should focus on getting some concrete tasks out of the retro points else the situation will always remain the same.

Here are some of the preparations that one can do before plunging into Agile:

1. Get in sync with market technologies especially testing tools like JUnit, Fitnesse, EasyMock etc. Moreover people should constantly strive for better solutions and so look out for new tools and new ways of improving processes, and look for new frameworks or new design ideas and patterns for recurring problems.

2. “Practices of an Agile Developer” by Venkat Subramaniam and Andy Hunt is a must read for every developer plunging into Agile.

3. Read “Craftsman series” on objectmentor.com and “Clean Code” by Robert C Martin. You will not give them much priority at first but after you get a handful of pain you will know your priorities.

4. Be open in stand-ups/planning/discussions, people value your suggestions. Say your opinion or ask for help if required, you are a stakeholder in the project.

5. Tests are not just for code coverage or quality measure. They also provide a sort of documentation, that is always up-to-date . People can look into the tests and see how to use this piece of code.

6. Automate everything in the build at the start of the project. If anything as small as Checkstyle gets left out then it will have the same fate as in any other model, mostly talk and very little action. When you realize this it is often too late for any damage repair techniques to help.

You have learned to speak A-to-Z so first unlearning it and then learning Z-to-A will not be easy. It will take its share of pain but after you are done you will know it is worth the effort.

So, Go team!! :)