Venkat Subramaniam on Pragmatic Agile Adoption

1. So Venkat can you tell us a little bit about yourself and what you do?

My name is Venkat Subramaniam, I'm actually a consultant and trainer by trade ; my passion is in training and sharing my ideas with others. I've been in the training business for more than about 8-9 years now. I go around the world teaching courses on various areas. I first started out with C++, moved on to Java and eventually to .NET.

My passion today is to do two things: one is to train people on technology; the other thing I've been doing is to work with teams in various companies moving them towards Agile development and how they can be successful in developing software. The other thing I do is I also teach courses at University of Houston where I'm an adjunct faculty at the Computer Science Department; I really enjoy working with the students and I've had the chance of doing that for the past 14-15 years. So that's pretty much what I do. My focus these days is essentially working with clients towards Agile development and succeeding in software development.

2. What's been happening with the extreme programming movement over the last couple of years?

The word "extreme" excited quite a few people because of the word "extreme" and the intensity towards that. It also scared a few people away from that as well in terms of "Oh my God, if it's extreme I'm going to go overboard". What is XP? There's a certain amount of uncertainty among people as well. Some people think that XP actually means coding and just mercilessly sitting there and writing code. There's a misconception that XP means there's no design and some people just actually use it as a way to apply their strentghs towards hacking, which is totally contrary to what XP is all about. It actually doesn't matter whether we do XP or not. What is more important is two things: first we should succeed in developing software, and the second is we make sure that we provide value to the customers, to be able to deliver software that actually works and solves their actual problem. XP has done a few things very well. It brought to us the fact that software engineering it's really not what it's supposed to be or what it claims to be. Over the years we've seen failures and we wanted to get success in that process, so we decided to use what engineers have used over the years. In doing so, we have gone towards more ceremonial approaches. We tried that for 20 years and we know that it didn't work. It takes out a certain amount of courage to stand up and say "Wait a minute, all these things we have been trying don't work maybe there's a different way to do things ". And I think XP did that for us. It showed us that we were not doing things correctly and that we should do things differently. And that's what excites me about XP. But by the same token, if anybody follows any methodology, doesn't matter what methodology whether XP or not, and if they follow it just because they have to follow what's been prescribed to them, I don't think they will succeed in what they do. So it's very important that organizations understand what it needs to succeed in software and then use what applies to their own projects. I don't think we can build one shoe that fits all and people have to figure out what size of shoe fits them.

3. How much is XP being used on the market?

It's hard to say how much people are using XP because a lot of those who claim to use XP, when you really talk to them you actually realize they are not using XP. We have seen that with almost every technology in practice we have out there. People claim to use objects but they really don't do good object oriented design; it's really no object oriented code even though they are using objects in it. In a similar way a lot of companies are using tools. I had a client I was talking to and he said they were using continuous integration. And when I ask them what they are doing with it I find out that they don't have any unit test and most of the people are not happy getting email notification when things fail. So they said that the way to fix it is to turn off the notification. People will claim that they have followed the practice but we have to really realize how much they have used. It's certainly nice to see that more people are becoming aware of this; more and more companies are beginning to realize that they need to follow. On the same token amount we've seen some companies which are trying to follow a more rigorous and ceremonial process. I really don't know if I can give you a number on how much XP is being used, but certainly more and more people are beginning to realize that they need to focus on succeeding and they are really looking for not just XP but other Agile practices as well.

4. You often talk about XP and Agile in the same breath, isn't XP just an Agile process? Can you do XP and not be Agile?

No. Actually in a rough way I would say that XP is to Agile methodologies as Java is to Object Oriented Programming (OOP). So it's one way of realizing agility. Just like where you don't want to stick to one language to do everything, I don't think you should stick to one practice or methodology to be agile. You could actually mix XP with other practices, other recommendations from other methodologies. To give an example: I really like mixing a couple of methodologies together to achieve the combination of things I want. To me XP is great; the 12 principles of XP ..., but when it comes to XP what does it say about keeping track of the project, estimating the project? There are a lot of programming level and team activities in XP, but as far as project monitoring, I see the "burn up" chart and the backlog in Scrum to be more effective. The stand-up meeting in Scrum I really find very effective. I think it's important to pick up the agile practices that we could gain from and pick and chose the ones that apply to vertical project and gain from them. Personally, I don't think we should just follow one methodology and say that's what we're going to have to use. We have to pick and chose what makes sense on the project and we got to strike the right balance on what makes sense on the given situation for the size of the team, the size of the project and all that can have an impact on what we decide.

5. You mentioned a couple of times that the nature of the project and the size of the team should determine what processes to use. Can you give us some example? What might a larger organization use versus a smaller one and how would you pick and chose the agile processes that you want to use?

That's a very good question. A lot of Agile methodologies and practices seem to really talk about small size teams and smaller size projects. We could ask ourselves "If my project is really large, what am I supposed to do?" There are a couple of ways to look at it. Number one: we have all seen that most of the large projects actually fail. For example Capers Jones talks about projects and he monitors 215 projects and he figures that only 10% of the projects actually are successful. That is extremely a low success rate and I don't think we can survive on the long-run with such records as in industry. Large projects particularly fail for a number of reasons. If we take off on a project for 3 to 5 years we aren't producing any value,we have constantly worked towards producing something, but we have no way of knowing whether this actually has value to the customers; and then when we do try to come to the customers and then when we do come to the customers and try to present it, they have moved on to other things. It's a dynamic world, it's ever-changing. I'm being brutal when I say this, but show me a long project and I'm showing you a project that's already failed. It's important that we take a large project - and to me managing a large project is trying to eat a large pizza, nobody in their right mind would try to stuff a whole pizza in their mouth, we eat a pizza a bite at a time, and in fact you bring one or two slices to your plate to start eating - in a similar way you can take a large project and say :"Ok , this is a project I want to implement, I'm not going to go at it in full strength, let me decide which parts of this project I can implement as a first set of goals and have that delivered 6 months to a year at the most, and let's get it out, get response from the people, get feedback from them and see where it goes." And I think we will go towards greater success doing that. So if we start partitioning the problem into smaller and more reasonable sizes, then we can start applying Agile methodologies towards it. And that's exactly the case with size as well. For example if you talk about what size is reasonable, most of the Agile practices recommend a smaller team size, 10 to 12 people, pushing it to maybe 20 people. What if you have a team of 30 or 60 or 100 people? You don't want to put a size of 60 or 100 people on one XP project and have them all work together, because the more the people, the more you're going to spend in communication than getting things really implemented. Once again "divide and conquer" plays a role in that. You can have one XP project with 60 to 100 people in it, but you have to really segment the groups into smaller teams that work much more closely; so there is a intra-team communication a lot more than inter-team communication. Then you can go about implementing the project. I think it's partitioning and "divide and conquer" that will help us towards attaining these goals and scale the process to a certain extent. Of course if the project is really, really huge, if you're a team of 700 people, of course it's going to be extremely difficult to even try to partition and achieve success in those. Unfortunately Agile methodologies don't scale up to that level. But I don't think any software methodology has been successful to that level, so it's not a problem of these methodology that they don't scale up, the size itself is the problem in those cases.

6. Let's say a team has no experience in agility, where do they start?

That's a very good question. If a team doesn't have any practice with Agile development, then what do they do? One way to answer this question is this: If my team has no experience in Java, and they jump into developing an application with Java, what do they do? Unfortunately we can't pick up things really quickly and become successful and productive with it. It takes quite a bit of effort. And we have seen this in the time when object oriented technology was introduced. A lot of companies were getting into it, a lot of people were claiming they couldn't succeed with it. And now we know that it's not the problem with object technology, it's a problem with people trying to adapt to it. In a similar way when we are trying to get into Agile development, we being successful as a team requires that we understand what agility is. Many times we are trying to implement certain processes, certain steps because somebody else is doing it or because somebody else told us. I think we should take a minute to understand not what steps we need to follow, but why we are following it. It's like watching somebody do something and say "Wait a minute. He's a successful person; he's doing that so obviously if I do that I'm going to be successful." That doesn't quite work and we know that from real life. So we need to ask ourselves "Why are we doing that? What's the value it provides?" And also if you follow another Agile methodology, take XP for example; if I want to become Agile and XP tells me the things I have to do, let me jump in and start doing each one of them and I will just follow them rigorously. That may or may not work. If we are just following it without knowing why we are doing it certainly won't work. Here is an example: I was called into a project which was behind schedule, definitely failing by all means and it was a project scheduled for 8 months; the company had spent 5 months on the project already when they called me, and they said "we're probably reached about 10% of the goals, we've got 3 months left and we don't know if we're going to be even remotely successful. Can you come and help us with this project and see if we can turn this around?" One of the first things I said to them first was that I couldn't promise success, that I could not do miracles for them. With 3 months left on the project, how am I going to make them succeed? The team was totally non-agile, they had no experience with unit testing, and they had not worked with any of the tools that promote agility. Sitting there I could have said "I know Agile is successful, so I'm going to put the team on Agile development, I'm going to follow XP or Scrum"; but I would have actually driven them towards failure very easily. To give an example of that, let's say a patient walks into the hospital, he's recovering from a heart attack he had a few minutes ago, and he's in bad with the doctors monitoring him, and the doctor comes to him and says "If only you had exercised, you wouldn't be in this case, so get out of the bed and get on the treadmill". The guy would die obviously in that situation. In a similar way, if a team is not Agile and they have 3 months left for the project to be completed, I can't ask them to do unit testing because the guys are going to take a month to figure out how they could become productive with unit testing. That's the easiest way to fail the project. So we have to apply a certain amount of pragmatics when it comes to applying any principle of practice. We need to take a pulse of what the situation is and focus on what provides the maximum value for the outcome. So to go back to the example of the project I worked on, I realized that given the 3 months we had, how can we promote interaction, how can we promote success in the project? One of the things I followed was a stand-up meeting. The team really did not have a clue what anybody else in the team was doing. When you go to somebody and ask them, "What is he working on, what's the status of the project?" usually they'd throw a blank face. "Hey I know what I am working on and I hope that's what somebody else needs". By bringing people together for a stand-up meeting every day, we basically brought people out into the open where they discussed about their problems. They knew what others were doing and also they were able to check back and say whether what they are doing that day or what they plan to do that day is relevant to the overall goal of the project. We also started having an iteration and a demo at the end of the iteration each week of the project; so we went every week at a lunch time where we did the demo of the project to the key customers, and showed them where the progress is, ask them what we should be doing next. They were able to give us feedback in what we were doing right and what we were doing wrong, and that certainly helped. So in that particular project in didn't go fully agile, but only decided to use 2 or 3 things that actually would bring out the maximum value. Incidentally, on that project we actually delivered the application two weeks ahead the schedule, and I consider luck being part of it also. The team worked very well with the ideas that we brought in. It definitely was a group of people who were eager to succeed. That's something that contributes a great deal. If the culture is not something that can adapt to this technology we will not have success as well. A lot of things influenced success. Just like how we say "Let's develop an application iteratively and incrementally", I think we should also get into agility iteratively and incrementally. Try things that provide the maximum value. Try to do tactical things and at the same time use strategic things. Gain success, because success is so beautiful; once you succeed on a few things, you get confidence and you go towards bigger things. I think that's a better way to proceed than saying "Alright guys, we're going to jump into this with everything in it today" and then we find that the water is too cold that we've jumped into.

7. On that last project you mentioned you worked on, what processes did you want to introduce that you decided were too much for that team?

If you ask me, one of the things I care and emphasize the most is unit testing and continuous integration. At the programmer level I would almost refuse to work with a team which doesn't do unit testing and doesn't have continuous integration. However, in this particular case, since we were so much behind schedule on this project when I was called in, it isn't practical going to unit testing and then still be successful on the project. So what we decided to do was to do stand-up meetings and a weekly demo. But there was another thing that I also noticed. One of the failures in software development is that we postpone integration. Integration is such a big risk in software development. And when I walked into this team I realized that they were building a certain important sub-system which was supposed to be integrated into a main system, and people have totally ignored the main system and integration with it for the past 5 months while they were working in it. And when they said: "Don't worry about it. When we get this done, it's a module and we can integrate it" the gut feeling in me said "This is not going to be fun, let me understand it". And when I found what was going on, I found that the code they were developing was totally incorrect when it comes to the model they need to follow. For example, to give an analogy: when you need to be developing panels, everything they were developing were frames. And imagine the amount of change you have to go through when you have to start integrating. I think that's where again integration plays an extremely important role and the more they push out integration on projects, the more we are increasing the risks. I think it's important that we take a pulse of what's going to really produce results, what's really going to produce success and work towards it, rather than following any methodology by the book. The people who are already working on the team can give you a certain amount of idea, especially if you are a consultant going into a team. You really need to get a good understanding of the capability and the competence of the team and what they can do. That is more important to succeed than just following some process by the book.

8. What are some of the lessons learned from Agile that we can use on large applications?

That is something interesting, because when you are dealing with large applications, typically a lot of things go wrong. You've done applications where I'm sure you spent several hours, if not days, trying to figure out something that has gone wrong - and why do things go so wrong and why do things take more effort to fix? And also typically if you noticed, let's say I'm working on a program and it's a large application, something doesn't work so I say "I've spent 4 hours trying to figure this out, what's going on?" Typically my boss would say "Don't worry I'm going to send Jane to help you!" So Jane and I have jumped on the project now, we've spent a few hours trying to figure it out; we have not had any luck. What does my boss do? He says "Don't worry I'm sending Bob to help you". So we let more people jump and pile on the project and we try to attack it in a way that this application has turned itself into a monster. And that indicates to us that we are doing something really wrong. In fact the approach should be quite the opposite. If something doesn't work, rather than putting more people to work on it, we actually have to start tearing the application to smaller pieces. I'll give you a couple of examples where I have learned my lessons on this. This goes back several years: I was working on a very large application, a very huge project. The project had been in development for quite a few years actually on that time, and I implemented a certain concept which was supposed to be extremely simple to implement; in fact, if I remember correctly, that was the first time that optimistic locking was introduced in this object oriented database. I looked through it and I said "This is a piece of cake. There's no big deal." It took me a day to modify the code and of course with confidence mixed in with a little bit of arrogance, I said "This is no big deal, I can do it". Once I completed the code change we started realizing there were a lot of problems; we had database corruption, it was a mess. We worked on it for several days trying to figure it out and we couldn't really find what the problem was. Whenever I called the vendor asking for help they would say "Why don't you send us the code?" There were a lot of problems sending the code over; confidentiality comes in, the code was too large to send to them those days, and basically I would just sit there and try to struggle through the problem, bringing consultants to help out; and it was a disaster. It took me about two or three weeks to figure out the problem and finally I found that the problem was not in my code at all. It was a ServicePack that we had to put in from the database vendor to solve the problem. I learned this lesson the real hard way. Well a couple of months ago I was at the client's site visiting them, and they said "Hey before you go, would you please come and take a look at this application? We've been having trouble with this; our folks have spent three weeks trying to figure out what the problem is. Why don't you take a look at it?" One of the things that come to my mind when I hear this is this: "You guys have spent 3 weeks on it, I have no clue what the application is. How could I ever look at it and solve your problem? Yes I'm a consultant but I'm not a magician." I went with reluctance; I said "Ok let me take a look at what's going on". And the programmer was showing me an applet that was making a request to a servlet and saying "Look over here, I'm entering these data, Punch in th This value, click... look over here it failed." And I was sitting there and said: "Sorry I haven't the slightest idea what this application does and what's going on here". What's going on? How can I fix this problem? So I said: "Do me a favor. Can you show me the code where the applet is making the request to the servlet?" So he brings in this function and shows me the code. I said to him to copy that code into a separate file. "Show me the servlet code. Copy that piece of code into a separate file." I brought that over to my laptop, wrote a little applet, wrote a very trivial servlet, about 5 or 6 lines of code for the applet, 2 or 3 lines of code for the servlet just to send the message and get back the response. I wrote it on my machine and it failed exactly the same way on my machine as it did on his. And I said "This is great. It's not working!" and he was amazed "Well that's why we've been facing for 3 weeks, and you're happy that it doesn't work?" I said: "No it's easy. If it fails on my machine and it doesn't have anything to do with application, I can put my time, I can understand the seven lines of code, I can fix it". It took me about one hour to fix this after that, but I was able to fix something for them that they had been struggling with for 3 weeks, not because I'm any better than them but only because I did not try to attack the problem in its entirety in their application. So, I think one of the key ways to solve problems is to isolate the problem from the rest of the system. We wouldn't obviously try to repair an airplane engine while it is in the mid-air. You bring it down, put it on the table and fix it.

9. What are some codes and techniques to isolate the code well from one another?

If you are really dealing with a large system and you just walk in and look at the system it is going to be very difficult to isolate it, there's no doubt about it. Why? Because the code has been built up over the years and we have started building complexity into it: a lot of coupling and dependency. Have I worked on projects where there are extensive amount of coupling in the code? Guilty as charged. Absolutely. We all learned from mistakes we've made over the past. So it is very difficult to take a system and suddenly make it easier for you to isolate things. But as we start refactoring the code, as we start writing new code, if we can pay attention to how we can isolate, that can be very useful. And one way to isolate - that's where I get excited about unit testing and mock objects - if I have to unit test a piece of code it is impossible to write effective unit tests for a method that's 300 lines. In order to be unit testing you're going to make the methods smaller. To give you an example: I really got excited when I signed up to write a book with O'Reilly; they sent me a CD by William Zinzer; the book title and the CD title was "On Writing Well"; Zinzer in his book, 25 years ago, talks about how you write well. He's talking about non-fiction, not about programming; he mentions four things about writing well. He talks about simplicity, clarity, brevity and humanity. Let's just focus on brevity for now because that's relevant to what we are talking about. In brevity he says it's better to write smaller books than larger ones, better to write smaller chapters than longer ones, better to write smaller paragraphs than longer ones, better write smaller sentences than longer ones, and smaller words than longer words. That just got me excited, because everything he was speaking about, my mind being my corrupt mind, was thinking about programming all the time. So when he talked about brevity, simplicity, clarity and humanity, I was jumping out of my seat. I thought: what he said applied very much to programming. And talking about brevity, smaller functions are better than longer functions, and smaller classes are better than longer classes. We're talking about the concept of "cohesiveness", so if we can make our code more cohesive, it becomes easier to write unit tests on them. And by making the code more cohesive and smaller by writing this unit test, we naturally attain isolation in our code. But what do we do when our code has a lot of dependencies? hen it becomes critical, that's when I think we can benefit from mock objects. Mock objects gives us less reason to give excuses for not doing unit testing. So things that I've learned in Agile development, like moving into isolation of code and dealing with larger applications, is an effective way to use unit testing and mock objects to gain that isolation. And even if you're not doing that; let's say you're dealing withbulky code, when you have problems, start ripping out the code, and isolate them and then when you find the problem you could refactor the code, and if it's not today, then down the road you attained a certain amount of isolation every time you touch the code. It's not easy but I think it's not impossible to achieve.

10. For companies that don't have a training budget, what are some of the techniques for them to be constantly learning and keeping their skills up-to-date?

We are in a profession where we are dead if we don't keep learning. To me being a software programmer is like being on a treadmill. You cannot afford to stand on a treadmill when it's on; you get thrown away. And that's exactly what happens the minute we quit learning in software development. Learning is something extremely important. And learning by taking a course, it's just one way of doing it. You could attend good conferences and symposiums definitely, but again all of those require budget, no doubt about it and we certainly must do that, but what's important is: that is something we're going to do a couple of times a year. That doesn't mean that we learn only a couple of times a year. So what do we do about learning? I want to talk about this as iterative and incremental learning. In other words we can not afford to learn something huge and large in just one day. We did this as students, cramming before the exam; we certainly shouldn't be doing that as professionals. So how can we keep up with technology? If you look at any technology that's out there, if you've been in a cave hiding for the past 10 years and you come out, you're going to see the technology around you and it's going to be overwhelming. Like XML - you don't know what it is, you don't know about a lot of things around you, but if you have just kept up with learning about a few things here and there, picking up a new technology it's going to be relatively easy. For example when I moved into .NET from Java it was extremely easy to do that because I've kept out with C++, com, Java and also the XML technologies. And what is .NET? It's a combination of everything I've just mentioned with a few additional things, so I just had to learn about what is similar to what exists, and what are the new things in there. And the change and the moving over were extremely incremental to me. And that's the way we should move. How can I keep learning? I think there are 3 ways each of us can learn. The first way we can learn is within an organization. There are really smart people who work with you and what you can do is organize your own brown bag session. For example let's pick Wednesday afternoons during lunch time. Everybody brings their own lunch. Let's say you're a team of 7-8 people and you say "Next Wednesday we're going to talk about concurrency in Java and John is going to be the lead in this particular discussion." John will open the meeting and he will spend 10-15 minutes giving a talk on concurrency in Java, for example, and then he would open up for discussions; you can talk about what concurrency is, why it is relevant to the current technology, why it is relevant to everything going on in the process or industry, or maybe why it is relevant to their own applications, what they are trying to build, where they need that for attaining a certain amount of scalability and so on. Then you can talk about things that are really specific to your application and domain. That is one advantage you have in learning within your organization is that you can relate it back to your own concepts and methodologies and a domain in your own organization. So having these brown bag sessions is one important thing. The other thing I would also emphasize - I mean the cliche of course, "don't put all your eggs in one basket", if you only know one language or one technology, then you're actually putting yourselves at an extreme amount of risk. Like Dave Thomas says: "Learn one new language every year". You may not use the language today. Let me give an example: let's say you've learned Ruby or .....; yes, you may not use it today but you get excited about learning a few new things. "Oh look at that closure; what's closure? That's a new concept, a different concept than what I've seen even though some old languages have had some trace of that. What's the DSL-like syntax?" Now when Java turns around to start implementing these kinds of constructs you don't have a learning curve because you know these concepts; you just need to figure out the differences and the nuances of how it's implemented, and you're way ahead of the rest of the crowd. I mean in today's market it is not important to be employed, it is very important to be employable. And that's one way to be employable in the future. The third thing I would recommend, for at least a good segment of the developer community, is the best way to learn something is to start teaching it; I can vouch for that myself; I've been in the teaching business for about 14-15 years. I started teaching C++ when I had no clue what it was and I learned a great deal by teaching. Teaching is extremely rewarding, and I'm not suggesting that you should give up your work and go and become a college professor, but explain to the people around you, be a mentor, go to your users group, give a talk, pick up a topic and say "I want to talk about this particular topic" and give a talk. I grew into a nation and a worldwide speaker by starting at my users groups. Actually my first time to ever speak was when I was speaking at a users group and one of the organizers of the conference coincidently came to my user group meeting, picked me out and said "Would you come to speak in my conference?" It can open up a great opportunity not only for learning but also in your career development. So I think it is important that we all spend time; if you are a person who is interested in a 9 to 5 job, I recommend that you have a career switch. Software development is not a field where you can just go to work; we are a professional community; it requires a great deal of effort and agility to keep up with it. So get on the treadmill and march along and that's the only way to succeed.