Rebecca Wirfs-Brock on Agile Design and Architecture

1. Hi I am Srini Penchikala and I am here at the Agile 2009 conference with Rebecca Wirfsbrock. Thank you, Rebecca, for the opportunity to interview you. Can you introduce yourself and tell our readers what you have been doing currently and what you are working on?

I have been doing software design and object design for twenty years and the last five or six years I have been involved in both Agile and non Agile projects at the level of design, architecture, designing flexible or adaptable systems and I still do that as well as teach and consult. I had to give up Smalltalk in 2000 my last Smalltalk client was in 2000 but mostly I have been working with people building enterprise systems, or products using Java, .Net technologies sometimes embedded systems so it's a pretty broad spectrum.

2. Speaking of design, does design have a role in Agile projects where products are fixed in time iterations and are fixed in scope, what is the role of design in those projects?

Well some people might say that if you are for example doing test first development that design happens between the key strokes as you are planning the next test to write, that's where it happens, right there. But I take a little bit different view in that I think that there are certain rhythms to design, when I am doing something new I may want to do some design thinking, hit the pause button on writing the test and thinking about structuring stuff that could have me explore some alternatives before I turn to the code. That code is the easy part, so I think there is a role for design and architecture and there are different rhythms for when you do it on Agile projects.

3. How different are design and architecture compared to just coding in an Agile project?

It's interesting, you ask how is it different than coding in that I think that when you are coding you are structuring a solution, so that is designing too but it's a level of detail that is not necessarily capturing what key abstractions might be, what the customer facing value is, it's providing a solution. So from a design sense sometimes we need to take a step back and if we believe in principles like domain driven design or responsibility driven design, start thinking about do we structure our code, do we structure a problem, structure a code in a way that matches the problem.

So design thinking at that point isn't quite always "I am not always just coding, I am doing some thinking and abstracting and sketching out and figuring it out". I think that a role of a designer on an Agile project who wants to be very customer responsive should also value reflecting their language into a solution, if they have the time. Now if they are banging out really fast and they are using Agile as another word for fast and furious they may not take the time.

4. Ok, that is very interesting. So if design has to be part of the iteration, are there any techniques that developers and designers can apply to integrate design into Agile iterations?

It's interesting I just recently wrote an article about CRC card and the Agile thinking tool, and my argument there was CRC cards are, for those who don't know about CRC cards, twenty years ago Ward Cunningham and Kent Beck wrote about CRC cards so their twentieth anniversary is this year and some people say "That was object design, old technique we don't need that anymore we've got TDD to do our way", but sometimes when you are thinking about a new piece of software that you don't know about, it helps to back one level up and say "Well, what are the responsibilities before I think about adding some new functionality?" Between the key strokes I might use a card just even to record an idea or to sum up rather than just focus on the details. So that is one use of the CRC card is this personal abstraction tool.

Another use might be when I have a new member coming to a team and I want to explain the design not just dive into the code, I can have that conversation just laying out the cards perhaps even talking about them, moving them on the table, same intent that the original CRC card was as well as accompanying the code so I have different views into my design.

And a third thing that I might want to do is actually do some kind of design modeling when there is a new area where you want to explore for a small period of time. So that is one example of an old technique using it in a natural way.

5. Are there any other techniques that you want to discuss in the design context?

One of the things I have done with Agile teams is have them at start of a new project or a major new release cycle so it's not just a sprint but some new chunk of work, where maybe there is a few hundred story cards that somebody has presented them, and now they need to sort them out, so they have an idea of some scope of work, a very simple technique of forming a team is sitting down and after they have been presented that, instead of just going into the planning game, to sit the developers of the team write where they think are the easy bits the design challenges, write for ten minutes and then share their stories amongst each other and it is interesting because they didn't get it's a forming team ideas some people have then posted these things on the wall as "Here is our initial ideas, how did they change".

But often times developers don't share design ideas; the loud voice and then if the rest of us aren't quite so quick to argue back, our voices get lost so this is a way of sharing and getting team building that is very useful at the beginning of a new effort. So that is one technique that you can use; again low technique that can get people to share ideas.

6. You mentioned earlier CRC also includes collaboration so what type of collaboration will product teams need within themselves as well as with the product owner in terms of improving the design?

Often times if you are just doing Agile projects you tend to not write a lot of stuff down it's the face-to-face communication. And just like story cards are a placeholder for a conversation, sometimes you need to capture enough information so that you can understand what's a good design approach or a good implementation approach. So rather than just having the tests that are maybe structured maybe users write FIT tests wouldn't that be nice if they wrote fitness tests rather than having the tests drive the design maybe they need to have a structured conversation of understanding the degree of variability that exists that I need to support in my design, and how it varies in very concrete ways with the customer or the business person.

So that then I can take and turn around and think about what's a good way to support it in my design and in my implementation? So I have been lately talking with people about and trying to do commonality and variability analysis which is free low tech idea of writing, again I like cards so CRC cards, I like those index cards, writing down what the variation is and a couple of concrete examples and then getting to the whiteboard sketching out how things vary so that I as a designer might put into play the concepts that are really driving the business rather than just designing it in any way possible. So, concept formation I am a big fan, so if I can represent the problem in the domain in my code or in my design using the language of the concepts, then there's the closer connections I am a big fan of Domain Driven Design and so this is one technique again that allows me as a developer designer to have a conversation with someone who may not just give me that in a nice neat structured way.

7. Ok you mentioned DDD which is definitely getting lot more attention these days. Can you discuss how DDD pattern fits into Agile design development efforts?

Well, sometimes it fits and sometimes it doesn't I am going to be pretty honest about that. I think that when there is rich behavior or varieties of rules, and it is not just a simple data validation access store retrieve database kind of application, when there is interesting behavior and it may vary, and there is complex rules, DDD really does have a place and that is where it is probably more important. So not every type of Agile application warrants that effort but when you find that there is complex business rules, complex concepts that are represented in complex data, that's where it fits and it really fits very well. Of course I am an object bigot, what can I say?

8. The DDD pattern that Eric talks about these days, the ubiquitous language which is mainly about communication those kind of patterns fit nicely into the Agile environment but then designing and implementing which domain models may not always fit into Agile projects because of timed boxed iterations how does this domain models take longer time. So in those cases what do you recommend? What do you say developers should do?

Rich domain models for those who don't know what that means is basically I want to have objects that represent business concepts and I want to give them behavior so they are doing things they are not just DTOs or data structures that are retrieved and then manipulated by some other code that doesn't have any name or whatever so these objects interact and collaborate and do things. So a rich domain model is something that requires object design skills so if a team doesn't feel comfortable doing that, that may not be the simplest solution for them. But if they do have comfort in object concepts and interactions and object design skills, then that's the first litmus test right there. If they do have the time to think about it that way, they need to get practice in doing it.

But the other complexity with Rich domain models is that you have to worry a lot about mapping that back to the persistence so you have to have that skill set in place too to do good mappings and that adds complexity. If I could just do SQL and just retrieve the data and operate on it, it could be simpler. So if I need the behavior and I have the thinking and practice skills of doing it, it can make it very nice and very testable because I can easily isolate these domain objects and put them to their ... and the mocking is very easy and testing is pretty easy.

9. There is also some times when the design ideas can't be implemented as planned, they have deadlines so that kind of results in some kind of technical debt so is there such a thing called design debt and how is it different form technical debt ?

That is a good question. One of the things that I think about technical debt is that it usually is design debt. So that is sort of my bias sometimes technical debt can cover more sins that just the design debt. Let me give you an example: a technical debt might be that I implemented using a technology that I was comfortable with but it is not going to scale for example, that is a technical debt it's not a design debt. And I say "Ok it's fine I used this only knowing that they have so many transactions, that's fine I don't have to worry about this", that is a technical problem.

A design debt is when I implement functionality but I do it in a way that has muddled or made it overly complex or tangled and that is different than not using the right tools for the right job. So I think that technical debt can be just a combination of using the tools incorrectly. Design debt can be something where I baked in a solution that doesn't have that nice clarity and integrity that I would like.

Both kinds of debt though are things that aren't always bad so I guess sometimes the customer verification in an Agile project, is this functionality and is this working the way I would like is more important, or "Am I getting the right way to use it?" And interact with the system is more important than "Does it have the integrity that I need whether it's a technical thing or a design sustainability". So I can do that as long as I recognize that I am accruing and it has a cost later on, that is good, it's sort of sweeping it under the rug.

10. One way to address those debts, whether it's a technical debt or a design debt is to include a refactoring effort into the iterations. Can you speak about design refactoring and what you may have done in the past in the projects that helped with the design refactoring?

Refactoring is something ideally that you should be doing at the most responsible moment, and that doesn't mean that I defer it until things are ugly, people have different senses of smell as you know, some people are more in-tune than others, so in some sense there is two kind of classes of refactoring those sort of hygienic things that I do like put on clean socks every day that's sort of a simple basic as I add new functionality, and then there are these more significant refactorings that happens because maybe even the nature of the problem is more complex so I have to restructure.

Or I let complexity or hacking or making it work predominate for a longer period of time or what it does is sort of a cruise so that it requires this major kind of refactoring work. So the easy one is the daily hygiene the more harder one in an iterative cycle I may have to have some kind of, I guess I design debt reduction stories that I introduce into sprints for that kind of refactoring. So the other ones are daily, they are below the radar if your team is working well but the other one your team will need to make visible because at some point you need to do it.

11. To make the design limitations more visible in the project do you think that the design retrospectives like the regular retrospectives will add any value to the team discussion?

So often times when you have project retrospectives they are just focusing on "What did we do, are we happy?" and they are talking about delivering value to the customer and the way that we work and they leave out design explicitly. "That's just what we do, it's part of what we do" and you may talk about "Did we write the right tests, did we expect this or that?"

I think that I wouldn't in a whole group talk about a design retrospective but I might want to take the development team particularly and have a short retrospective after an iteration for two reasons one time if for example we didn't hit the velocity we expected and so our estimations were way off that's the time to say "Maybe we were working on some new core stuff that we really didn't estimate correctly because now we have deeper understanding". Or it might be that we were doing something that added more variability than our nice design was able to accommodate and we need to restructure the problem.

That was a design spike cause and we'll say "Ok that was fine but if we don't reflect on why we missed our targets that's one place to do it is to have it when things are at the end. It's not always when things are bad though so I may want to reflect on "Why are things going so well from a design perspective". And it could be that the refactoring work we did enabled us to go further and faster. So I think from point to point not every retrospective needs to have this but a team if they are design-aware might also want to say "Ok, we got stuff done fast, but how much debt did we accrue doing it?" So just a bit about why was our velocity the way it was and this kind of discussion makes a team for a few minutes, it might lead you to a different behavior the next iteration.

12. Going back to the commonality and variability analysis you discussed earlier, how can a project team manage commonality and variability when they are running really fast with the iteration deadlines? How would they know that something is part of the commonality part of the design, what's the variability, because the first time they would know is when they are working hard against the deadlines?

It's interesting if you are working hard against the deadlines abstract thinking doesn't do any good, it's like you can't force thinking about the structure of the problem when you are running fast and not pausing. To do this you really need to say "I need to understand the problem". Scott Ambler talked about Agile model driven development where he says "Well, there should be some box of time where you have to think about this usually you can vary based on user stuff or any changes to what we have already done that are going to be just slightly disruptive, in the functionality, so ideally it's something that you have to plan in.

It doesn't have to take a long period of time but usually when there is something new, and you want to know before you go. So even from a fast pace, something new that's a good opportunity to do it. And if there is something that is a variation or a deviation from what you have done, and you still want to go fast and you are worried about making your velocity, then there's a time to pause and have that discussion too. I think that in the XP world people use to talk about pairing for about ninety minutes and switching, you can in a ninety minute discussion this is not an up front variability analysis with the business stakeholder learn a lot if you do that but you have to find the right time, the responsible moment to do that.

13. Speaking about design and architecture, if you talk to Neal Ford he suggests that design should be emerging and architecture should be evolutionary. Do you have any thoughts on emerging design and evolutionary architecture concept?

Well design always emerges and that's why we refactor and re- think. I think merging design is not the same as design and re-design always. Which is sometimes you just say "Well whatever happens, happens it will emerge from the chaos will come some structure and order". So, emergent design is just something that recognizes that we need some time to structure our thinking we need to prove it out, and we need to be opened that we need to change or refactor at the most responsible moment. My words of advice about people who want to think about emergent design is that it is not a replacement for deign thinking at the most responsible time so there is still places at it that you might need to take things and put them aside and think about it and try alternatives.

Which gets down to architecture- architecture is nothing more than a strategic design decision in my view that are going to have fundamental impacts if we screw them up. So for example if I am doing in an internet based, deciding to do something in an Ajax way is quite different than just the static. If I don't know how I am going to handle more dynamic interactions than that's going to be really chaotic to the structure of my middleware kind of code and just say "Now I want to be more Ajax like". if I wanted to have something that is highly transactional, it just doesn't emerge so I have to think about "Am I going to use some kind of messaging structure or queuing mechanisms and how am I going to do that in a way that will support the to support what I need?

And it's not just my code it's just the things around it in the infrastructure. On projects that have significant amounts of architecture decisions in an Agile world it's not always that architecture emerges one story card at a time. I think that the most responsible moment might be taking aside and figuring out some of the hard bits just like the user experience folks these days, user experience designers, are recognizing that there is a cycle of discovery and confirmation and then there is this more regular cycle of doing and filling out of the details once you figured it out. Some of those things need to be done in a side band from the regular beat of all the rhythms of an iterative sprint if you will.

14. Sounds like Agile design discipline is very interesting and very important in the Agile development iterations. What are the qualifications or the guidelines that you recommend for someone who can be a good Agile designer, who is working on these Agile projects?

They have to be a team player, first off, and often times back in the old days people owned things, I owned my bit of code however I figured it out and in an Agile team it's a collaborative effort. So I have to have a give and take it's not my design it's the design we are evolving together, and they have to communicate it's not just "I am going to go in the next time through, hack it out and refactor my way". So you have to be for one thing a good team player and I think that another skill is, that's fundamentally there, certainly knowing good daily hygiene principles for keeping the design clean are very important so refactoring, doing the testing that makes sure that the design is testable all these kinds of practices are things that I think an Agile designer really has to worry about a lot.