I didn't start life planning to be a software engineer. Actually I was on a trajectory to become a mechanical engineer when I fell into the big software bucket. IBM had me even just a couple of months even before I got my bachelor of mechanical engineering and that was it. It was very difficult for me to extirpate myself out of big software bucket. After all, in 1970s this was where the money looked like it was.
I worked for IBM, actually a very brief time, I then did a bit of research at the French Institute of Telecommunication, learnt more software there and also I went to Alcatel, where I learnt a lot because it was the beginning of using computers and especially large networks of microcomputers to punctual telephone switchers and all kind of things in telecommunication.
That was a very important learning experience in terms of building large systems with large amounts of software, with a certain number of "-ilities" like "availability" and "reliability" and that kind of thing.
I then moved to rational software and I was primarily a software consultant working with large command and control systems, as a software architect, all around Europe initially and then somewhere around the Pacific for a while, until I ended up in Vancouver as the chief architect for the Canadian traffic control system. When I got out of this contract, 5 years later, that's when I took over the development of what was originally called the "Rational Approach", but became known as the "Rational Objectory Process" and then the "Rational Unified Process". That I did for about 5 years until the acquisition of Rational by IBM at which point I left and reached my ultimate level of incompetence - on the Peter Principle - which is teach undergrads.
I do not see them totally opposite to each other. They were born in very different context, with very different types of projects in mind, but the Unified Process can be used in a very Agile fashion, it can be used in a very stupid fashion. You can use it in a very completely absurd fashion. Somebody once told me "Oh, yes, inception. This is where we write the requirements and the elaboration - this is where we do all the design and construction - this is where we write the code". Yes, you can use it in a relatively absurd way.
I think that the RUP embodies many of the Agile principles. It doesn't have the same focus on the people aspect that we've learnt from some of the Agile methods, but in terms of iterative development bringing some discipline into this development process, bringing up a process that can be adopted to the various circumstances something that a lot of people fail to notice and do.
They try to use RUP out of the box "Oh, my goodness! There is 27 goals defined and there is only 6 of us! How are we going to do that? There are 37 work products defined, how can we do that?" I don't think that RUP was meant, by any one of us developing it, as something where you open the box and try to do everything that's written in it. It was more like a catalog of practices out of which you extract what you need for a given project. I have the same feeling with some of the Agile method. They were born in a given context in which they work pretty well. When you apply them in a context that is relatively similar in terms of the type of project, the size of team and so on, they work pretty well.
If you try to extend them completely outside of the context where they were born, that's where you start having some difficulties. They become tools "in such a problem to solve". Very often people hope that by doing all the practice as defined by the author, whether it's RUP or XP or Scrum, they are going to solve all the ills in their project. Actually, I think you need to do the opposite - ask what is it that we are suffering from? Where could we get some improvement? Then bring elements of processes and practices that would solve the problem that you really have.
I learnt software in a relatively big complicated technical systems, where we had large teams, and in some cases, a system that was safety critical. You have to have some discipline, you cannot just imagine that it's just a matter of speaking to each other and doing pair programming and having all the design emerging as a matter of just communication and refactoring. You need to have some predictability; you need to be able to demonstrate that you've reached formerly some defined level of quality. I think it's totally compatible with some of the Agile principles of communication and iteration and feedback loops and learning from our mistakes and learning from our progress and so on. You can marry the two. I think that it's very unfortunate that Barry Boehm named this book Balancing Agility and Discipline, as if they were in opposition. I think you have to be relatively disciplined in order to do Agile development in a successful fashion.
Oh, yes, that's another issue where each will depend on your context. There are many projects out there, probably 80 -85% of the software development project - there is an architecture is in place in day one, when you start the project all the big choices have been made, language, platform, framework - you name it. The concerns about architecture are probably only applicable to a small fraction of software development projects out there. That's probably one of the false dichotomies, where you have to be Agile or you have to focus on architecture. You can do both, when necessary.
Architecture is about making some important decisions that we'll have some long lasting effects on your system. If you ignore architecture and rush into software development by trying to drive software development solely with the functional visible part of it, you are running very rapidly into technical debt. Certainly, you may reach the point at which you have a first release, but if you have no focus on architecture, if you've always dismissed architecture as "Oh, this is big up front design" and "you aint gonna need it", "let's defer to the last responsible moment", you'll reach maybe that first release and after that you are going to suffer from a lot of technical debt.
Many of the architecture decisions that you've made - or the ones that you've not made - are very difficult to catch up later. Somebody, not necessarily everybody on the team, has to keep a focus on architecture and most of the architecture difficulties are related to quality attributes rather than functional requirements. It's how do we achieve security, how do we achieve scalability, how do we achieve maintainability, how do we achieve high availability of the system - that's where there will be a lot of difficult decisions.
Driving solely a project, a-la Scrum method, let's pick out of the backlog some good stuff that we can have a demo at the end of the sprint, may lead you prematurely into a lot of the technical debt. Again, it will depend on your context. There will be context where an architectural focus is important and there will be some contexts where it doesn't matter we have an architecture and it's probably good enough for whatever we want to do.
Yes, that's one of the key messages I'd like to get out. Real estate people in North America say that there are 3 important things in real estate: location, location and location. I think we have to focus on context, context and context. The idea that we're going to find some day some brilliant software developer who is going to define the ultimate method that's going to replace it all - I don't believe that!
The spectrum of the type of software development that we do all day in industry is so vast, it's so different in terms of business domain, size of project, the difficulties or the quality attributes that we're going to run, the complexity. It's so vast we'll never define the one method that will supplement them all. We have to learn from the methods and practices that exist, but we have to understand what do they solve; and when we are moving to defining a new project, we have to understand what out of whose methods we want to use.
Maybe we want to pick RUP like life cycle and maybe we want to do some pair programming and maybe we want to have test driven development and maybe we do not need architecture - it will depend on the context. There is a wide range of projects.
The one the we see a lot is web based database, web server, access through the web, they share a lot of characteristics. This may be the "sweet spot" for many of the Agile practices. I'm still very concerned about what's happening when you're not in a project that fits the sweet spot. The very large projects (there are still a few large projects being done around the world), the projects that have embedded software, the projects that are safety critical, the projects that have some very stringent performance constraints - those require a lot of attention. They are not going to just gradually emerge out of weekly refactoring.
6. Where to from here? Where is the software industry headed? Where is Philippe Kruchten headed?
If I knew, if I had the crystal ball there, that certainly would help. As I said, I think we need to abandon the idea that somebody will invent the one process that will supplement them all. We have to learn what's good and what applies in which context. We have to learn also that - and this is something that the Agile movement taught us - the key and dedicated ingredient in software development is human beings. That's the "machine" behind it. We're not going to build a software factory, shove in requirements, get software out of it, although it's still the dream of many people when you speak to the model driven crowd, they try to raise the level of abstraction.
Pretty much we've abandoned the idea of having the one ultimate programming language that will supplement them all - like PL 1 or ADAR 40 and 30 years ago. I think we have to admit that there are different aspects of software processes that will address different issues.
We are going to face some interesting challenges: for instance multicore systems are probably going to have to change paradigms. We don't know exactly how to exploit a large amount of parallelism. Also, the software that we develop and most of our mental model for software, relies on a few sequential threads, maybe exchanging a few things here and there. Multicore is going to change a lot of things.
We see more and more software being composed out of existing software. As SOA is just one useful paradigm or pattern to compose software. There will be more and more people composing software out of existing bits and pieces and gluing them together than, people actually writing code in C++ or Java.
Are the methods that we have now are adapted to this new paradigm? I don't know; we are not there yet, we continue to evolve, we probably find different programming languages, maybe things like functional languages, domain specific languages to address different types of issues as we continue to evolve.
The Internet is changing also a lot of things. There is this "information overload", we need to find other ways to organize things in a relatively complex system or some families of systems working together to reduce the information overload on the end user.
So, I don't know, I don't have a crystal ball, but it's still pretty interesting to look at what's happening. Personally, what interests me is what's the decision process, how do people make decisions, what's the reasoning behind.
I think we're very far from the traditional engineering paradigm where the engineer enumerates the problem and the criteria for the solution and then enumerates alternatives and evaluate the alternatives and picks the best one. That's not exactly how software development is done. It's a lot of decisions that are just made in an instant based and experience. That's something that is very interesting and where there are very few things that have been explored in the context of software development. It's interesting because there is some continuum between requirement elicitation and how we understand what it is that we want to solve. The work of the business analyst and the product managers and product owners and the work of the architect - we tend to see them as very different, using different contexts and paradigms.
The more I look at it, the more I see people doing the same kind of things, making choices and decisions. They do not express them the same way, but when you make a decision at a requirement level or at the architectural level, it's a little bit the same thing. What becomes a decision at one level, becomes a problem to solve at another level, which will end up being solved by making a decision, which becomes the problem for the next person in the chain.
The whole thing I see as a continuum. This is among the things that might allow us to reconcile the various aspects of software development. We're not having disciplines that are silo'd, business analysts doing their own thing with their own language and their own conceptual models and software designers doing their own things and so on. The two things may be reconciled in some ways.
7. What is the focus (and perhaps you've answered this already) of your personal research at UBC?
It's very difficult nowadays to find funding, especially government funding for things like software engineering, the part of software engineering that interests me, because it's too soft. Software process, if you speak with people from funding agencies they say "We funded software process for the last 30 years. What do we have to show off for it". I'm still interested in architecture and I'm looking at architecture on two different aspects - the decision process, architecture as a set of decisions, not architecture as a set of UML diagrams (I think this one is more or less solved). By looking at architecture from a set of design decisions, we go into the "how are these decisions being made", what's the rational for this design decision, how are they connected one with the other. We go more into the psychology of software designer and business analyst and that kind of thing.
Another aspect which is related to architecture, where I have a little project with the software engineering institute is technical debt, which is the unfortunate negative dark side of not having made the right architectural decision. How do people conceive of technical debt? Can we put a cost, a price on the technical debt? What are the strategies and things to get you out of the technical debt? -This is an area that seems to not have been explored very much except by Steve McConnel and Martin Fowler in little write-ups. There is very little actual research on technical debt.
Another thing that interests me is more on the technology side, the use of touch surfaces for many, especially in software development table top surfaces and big screens that you can interact with [using touch interfaces]. I'm looking at it in the perspective of large distributed teams working collaboratively together across multiple locations and time zones. It provides a good set of interesting research questions and fortunately I have government funding to do that because it directly supports the local Canadian industry.
The theme of SDC this year, and I'm invited by Software Education and you Shane and Martyn Jones, was a little bit far from my own personal interest - business analyst. I'll try to do my best to merge my interest with the theme of the conference. I'll be doing a workshop on software architecture for business analysts. What I wish as a software architect, the business analyst to know about software architecture. I'll also say a few things about technical debt and release planning and how to mix the good things and the bad things, the architecture and functionality and fixing defects and avoiding technical debt. My own little twist on the delicate and subtle art of release planning and iteration planning or sprint planning to speak "Scrumish".