Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Interviews Tom Stuart on TDD, BDD and Ruby

Tom Stuart on TDD, BDD and Ruby


1. [...] Tom you are doing a talk today on Impossible Programs, could you just very briefly explain what that's about?

Mark's full question: Hello my name is Mark Collins-Cope and I’m pleased to have Tom Stuart here with me at QCon London 2014. Tom is the author of the book “Understanding Computation” published by O’Reilly and is currently a freelance software engineer, and is in general a computer scientist. Tom you are doing a talk today on Impossible Programs, could you just very briefly explain what that's about?

It’s about the things that computers can’t do: so there is a temptation to believe that essentially any problem can be solved by a computer through the application of enough ingenuity and engineering effort, and my talk today is explaining that some of the things you might want to do with a computer are actually not possible. I mean some of the things that aren’t possible are quite esoteric and mathematical, but there are a lot of other consequences of those things that relate very much to the practical things you want to be able to do with computers. So I just wanted to talk a little bit about what the limitations of computation are and how those limitations affect the things that you can do as a computer programmer and software engineer.


2. [...] Do you think there are sometimes theoretical aspects of computer science that are ignored too much in industry?

Mark's full question: I think people can find a link to the talk on the website here. So, when I was looking at your talk that you gave at a previous conference, some of the things that you were saying in there reminded me of some of the more mathematical and formal specification type aspects of computer science - program proving or Design by Contract; that sort of thing. These things were kind of popular not so long ago but seem to have faded out of interest, in terms of industry at least. Do you think there are sometimes theoretical aspects of computer science that are ignored too much in industry?

Possibly. I mean I think the perspective I have on computer science, or my personal interest in computer science, is much more related to it being just fun and interesting. For me it’s an interest in computer science that keeps me enjoying my job as a software developer and allows me to kind of keep being interested in the stuff that I’m doing; it's thinking about all of the more mathematical connections it has with other things and how things are really working under the surface, and so I think that there has been... I think it’s a shame when people don’t know anything at all about computer science, if only because it’s just generally an enriching thing. I mean in particular that issue about program proving and things like that. I mean people have tried all kinds of industrial approaches to doing things, proving properties of programs in industrial settings, and none of them have quite succeeded or taken off, and I don’t know exactly why that is; but I think that there is not a huge appetite among people who are working with computers every day to need to deal with systems like that all the time. But I think that that’s probably separable from the issue of, "Am I able to learn interesting things about computers and kind of find out what it means for a computer to run a program or what it is that programs can do?" which is what I’m talking about in my talk; or like various techniques that you can do to make programs go faster, or make them smaller, and all of those kinds of things.

I think that people in general would enjoy their jobs more if they had more of a ... if they could get more enjoyment from the pure material exploration of computer programs as a thing that you worked with and not... It’s very easy when you are a programmer to get bogged down in the kind of boring details of writing computer programs and you’ve just got, you know, core dumps, and failing tests, and you've got to reboot your machine and stuff like that; and for me it’s the sort of slightly higher minded interest in what is happening inside the computer that makes me able to kind of coast through all of those less enjoyable episodes.


3. Do you think, for example, the things like formal program proving, although they may not be realistically used in industry, do you think actually learning how to do those things can add any value to someone's ability as a software developer?

I think so. I think having an awareness of those kinds of techniques can help you to think about the programs you are writing in a slightly different way. In particular things like program proving and Design by Contract and that kind of thing are sort of predicated on the idea that a program has a meaning; that when you write down a computer program you are sort of writing down the denotation of something slightly more abstract; and I think that in general you don’t otherwise tend to think like that about the programs you are writing.

It is very tempting, especially when you are working in dynamic languages, because you don’t even have a type system that is looking over your shoulder, you are just hammering away at the keyboard and writing code; and sometimes it’s got syntax errors in, and when it doesn’t have syntax errors in, it’s got bugs in and things like that, and so I think it is nice to be able to take a slightly more abstract view of what you are doing and to think about...

You know, Design by Contract wants you to think about programs as kind of pieces that are designed to slot together, but the interfaces between them are sort of separate from the code that lives inside those pieces and that’s a very useful mental model to have of a program. Even if you are not actually doing the automated program proving stuff, it’s still really handy to just have that model in your head of what you are doing, so that you can kind of help make sense of the program that you are writing.


4. [...] Do you think there is a trade-off between those two things; are they contrary to each other or do they support each other? How would you see it?

Mark's full question: And coming back then to Design by Contract, I mean that became popular in Eiffel - that's where it really became popular - and it was built into the language and there were various copycat libraries in other languages: C++, Java, and so on and so forth. But since the advent of Test Driven Development again that sort of approach, or that idea of having pre-conditions and post-conditions to check the validity of a particular method or procedure, seems to have gone away a bit and the emphasis has gone on having external tests to the actual method or procedure. I mean do you think there is a trade-off between those two things; are they contrary to each other or do they support each other? How would you see it?

I think they are probably complementary. I think that essentially anything that you can do, any method that works for you to help you and the computer to pay more attention to the content of your program, is a good thing. I mean languages like Eiffel, as far as I’m aware, haven’t taken over the world and probably it’s fair to say that methodologies like Test Driven Development have been more successful than programming languages like Eiffel. I think that ... There is this great thing that Simon Peyton Jones who, if anyone is, he is responsible for the Haskell programming language: so he is like one of the leading lights of the, at least the statically typed programming community. He has this thing that he calls “the region of abysmal pain”, which is the difference between what the..., he talks about it in the context of type systems, but in general it’s the difference between the kind of programs that the computer can see are going to work properly, and the kind of programs that you know are going to work properly. And every time you as a programmer sit down and you write a program and you know, because you've thought about it, that that program does what it's supposed to do, but the computer disagrees with you because the type checker doesn’t work or because the Design by Contract stuff doesn’t match up, or whatever, that causes frustration.

And so I think that there is space for formal techniques and type systems and things like that that allow the computer to have this kind of rigorous influence over the activity of programming, but there's also a demand among programmers to, more or less, be able to just write the programs that they want to write; and at the moment the formal techniques and automated techniques that are available for program checking are just not advanced or not sophisticated, or at least not usable, enough that your kind of archetypical ordinary programmer can just sit down and write a program and have the computer agree that that program works. As soon as you start trying to do something more sophisticated than just, you know, adding numbers together, suddenly you have to start understanding either contracts or complicated types or whatever. And there is always going to be...

Methodologies like TDD allow you to think about your program in a different way, which is that you are not being asked to satisfy any kind of formal constraint of your program. What you are being invited to do is to exercise your program and to provide a kind of a second client to your code. So it’s not just going to be run in production, it’s also going to be run in a test environment; and it’s not just going to be used by a human, it’s going to be used by automated tests. And if absolutely nothing else that just exercises all of your code in a slightly different way: ideally in a way that you might not expect a person to do it. And so, best case scenario, you would use a combination of those techniques; you’d have some kind of formal verification and you’d also have this kind of informal verification that automated testing provides and that would make you super sure that the programs that you are writing were correct. In practice I think most people probably do one or the other and sort of hate the other thing.


5. [...] Although there's no 100% guarantee because you might have tested for the wrong thing, but the probability is reduced. I mean does that work with you?

Mark's full question: It always struck me that the motivation behind testing is to do something twice, which is you do it once in the program and get a result and then you do it in another way, hopefully in a test somewhere, and if the two results match up, then you are fairly confident you’ve actually got the right program. Although there's no 100% guarantee because you might have tested for the wrong thing, but the probability is reduced. I mean does that work with you?

Yes, that make sense; and I think one of the... The way of thinking about it that makes sense to me... Because there is a temptation to, sometimes if you are... I remember when I started out trying to do automated testing and I was very much just writing tests that checked that my implementation was my implementation, and that is not very useful. Like there's a bit of an art I think to writing automated tests that actually provide value, which is sort of what you've said really: that you want your tests to check that your code meets the specification. And so a lot of people, I mean particularly in the Ruby community where I’m from, people talk a lot more about specifications than about tests; and we talk about Behavior Driven Development rather than Test Driven Development and we talk about examples rather than tests and things like that; so about using tools like RSpec and Cucumber and things like that to try and do this sort of 'specification by example' thing, where you're actually trying as hard as possible to separate the intent of your test, or your specification, or your example, from the actual implementation of that thing: because there is no point in writing a suite of automated tests that are just checking that you wrote the code that you wrote, because you already know you wrote it.

What you want to know is whether it does what you wanted to do, and so there are a variety of different kind of mind hacks for trying to trick yourself into writing tests in that way, and some of those revolve around the language, the human language, that you use to talk about your tests; and some of it revolves around the tools that you use, and some of it is like the process and like when you write them and who writes them and all of that kind of thing. And all of that stuff hopefully forms a useful input into the system of like, "What do we know about what this program's supposed to be doing?" And the more of that that you can pour into the computer so that you are thinking less about just checking that these two variables get added together and more about, "Well what do we actually want to happen?"; like, "Should those things even be added together?" and BDD would take it a step further and say, “Well, who cares about whether those two things get added together; what is the business value? Who is the stakeholder who cares about what happens? Have you asked them what should happen?” and sort of trying to push as much of that knowledge as possible down into the system.

So it’s not just you sitting down writing a method on an object and it doing some stuff with local variables, but actually it kind of gets pulled out into this big long system of like, “Who's this code going to be used by? What do they want? If they want that then what needs to happen? And what actually happens?” And you know all of the tests that you are writing are kind of adding value at every layer because they are checking things that aren’t just intrinsic properties of the code, but more sort of extrinsic properties of where the code is going to be used and who it is going to use by.

Mark: So I guess, summarising that, you are saying that if you just use TDD on its own, you still have to work out what it is you want to test and things like BDD are actually helping in getting to solve that problem, rather than the tests help exercise the code but they are not going to tell you what you need to write in the first place.

No and that is part of the idea I think. That reminds me of the interactions between static type systems and code, and tests and code, because a static type system is, in a language like Java or C++ or whatever... One way of looking at that is as a sort of a test suite that ships with your language that when you download GCC or the Java compiler or whatever and you ask it to compile a program, it's already got some tests built into it that are supposed to pass for every program in that language, so the stuff...

If your program in Java or C++ or even Haskell is well typed, that just tells you that your program is, sort of, not making any stupid mistakes and that is an amazingly useful property to have. But there are other properties of programs as well and there is no real formal method, or at least no automated formal method, that is going to be able to tell you anything more than that. So by writing tests and embedding into tests information from the world about what it is that's supposed to happen, and who it is that cares about this thing happening, you can definitely I think bring more information in than it’s possible for something like a type system or a compiler to ever verify about your program. Because, necessarily, whatever is built into a compiler, whatever it’s checking about your program, has to be suitable for every single program that you could write in that language, and so it’s not going to be able to tell you anything useful about the specific case that you are trying to address by writing that code.


6. Final question on TDD. Do you ever find yourself secretly writing the tests after you've written the code, and has anyone caught you ever doing that?

I think probably everyone who does TDD does that sometimes. So sometimes I do that just because I’m lazy like every other programmer. The defensible circumstances in which I do that are when I’m trying to do something which I don’t know how to test, and that happens a hell of a lot. Like a lot of the time the code I’m writing is doing something that's outside... I really like experimenting just for fun; writing code that I haven’t written before to do stuff that I haven’t tried; and so a lot of the time that means I’m writing software that I don’t know how to test because it’s doing something weird.

If I’m writing some software to generate some graphics or something, it may be that I don’t even know what I’m expecting to see, and I think that is a very different activity from when, if you are just a software engineer and you’ve got a stakeholder and a delivery manager and all of these people who are involved in the project and they’ve already had lots of discussions about what needs to happen, which hopefully you’ve been involved in, it makes a lot of sense to write tests and make those tests pass; and like that is the way I prefer to work. But sometimes when you are just exploring or playing or trying something out and you don’t even know if it’s going to work, or you don’t even know what is going to happen, it's sort of impossible to write tests first. And so the short answer is, I definitely have been caught not writing tests first, but sometimes I can give a long-winded justification of why I think that is acceptable.

Mark: Ok, so moving on to... obviously you're part of the Ruby community and something of an expert in that. I mean Ruby is a really interesting language and very elegant in many ways, but do you think it’s an appropriate language to write all types of application in? For example, given its dynamic nature and lack of type checking, do you think it should be used for things like life critical systems or you know, say live real-time systems or things like that? I mean are there limitations? I mean specifically not just because it's Ruby: maybe for other dynamic languages as well.

Possibly. I mean I think... My natural reaction to that is that I don’t really think that any programming language is suitable for life critical applications. I mean in particular for things like avionics control systems and nuclear power stations and things like that. My understanding is that for those systems they put them out to tender and they get more than one of those systems to be built, so you have two or three or however many systems, then they all have to vote on whether the control rods get lifted out of the reactor core or whatever. So there is a sort of a slight sense where I don’t...

It feels to me like a lot of the problems that might happen in life or safety critical applications are more to do with human error at the specification level than at, sort of, stuff at the implementation level that might have been caught by a type system. So I'm sort of slightly deflecting the question, but I think... that is my first thought. It’s that the most dangerous thing in any application is the collection of humans that have been responsible for deciding what that thing was supposed to do; and no programming language has as a feature that it will cancel out stupid stuff that you try to make it do. So that is one issue. I mean certainly in the case of things like real-time applications... I mean I wouldn’t use Ruby for real-time...

Ruby can’t give you any kind of the guarantees that you want in a real-time system, so it’s just not suitable. But in general, I think that it is a suitable language. It’s got, like any other dynamic languages, it’s got some sort of weird bits. Inherently it’s not as performant as writing Assembler, or even C++, or even Java. Whatever it is that you want to do, you need to strike a balance between the expressiveness and pleasantness and just overall niceness of the language that you are using, and also how well it fits your problem. And so if what you want to do is write a videogame that you want to run on Windows and Linux and on the PlayStation and the Xbox, then writing in Ruby is probably a stupid idea; you should write it in C++ and you should use DirectX, or Direct3D, or whatever it is that people use.

The most important thing is to pick something that's fit for purpose. But most of the time, most of the software I end up writing whether it is... I mean a lot of the time I’m writing web applications and all the usual stuff that people use Ruby for; but a lot of the time I’m doing stuff that is not web based and I still find that for most kind of practical applications Ruby is still a nice enough fit that I can enjoy the benefits of it being dynamic and it being relatively pleasant to use. And you know, the tooling and the ecosystem and all of that stuff sort of compensate for the fact that my program runs a bit slower than it would if I'd written it in C++ or whatever. I don’t know that I'd want to be on a plane that was being flown by one program that was written in Ruby though.

Mark: Or C++?

Or C++; no.

Mark: So a final question for today. I’ve heard it said that languages, because of their dynamic nature, languages like Ruby where for example you can intercept a method call, dispatch it to something else and then come back, and do those sort of tricks and things, or even build functions on the fly, that they couldn’t possibly be type checked because of that dynamic feature. It strikes me that probably a lot of the time, you tell me, that you're not actually using those sort of features of the language and perhaps you could just... couldn't you just have more strong type checking for most of the time, but have the ability to switch it off when you want to do something that is really clever.

Yes; I think that in research terms that is eminently doable and there are plenty of people who are doing research into type systems that allow you to have kind of partially checked programming languages, where the parts of the program that are doing things that the type system can understand can be comprehensively type checked, and then the parts of the program that are doing weird, dynamic stuff like you mention, are just kind of out of scope of what the type checker is doing. In terms of actually having a different programming language - because in principle you could take stuff out of Ruby and kind of make it less dynamic, right? So that you could make it more amenable to being type checked, and I think that is eminently doable. I don’t actually know anything about that, but I do know that like with Python for example there is a restricted subset of Python that they use in an implementation that can be type checked and where type inference works, and stuff like that. So Python, and other dynamic languages have the same problem, so it is totally doable. My kind of unjustifiable gut instinct is that there is probably... If you did that to the language, there would probably be a large enough set of things that it made difficult that you would have trouble persuading anyone to use it; just with the explanation, "But it's got a static type system".

What has propelled Ruby to the current level of popularity has been Ruby on Rails, and Ruby on Rails depends very much on being able to do things like dynamically define methods at runtime. And so if you, even if you personally are not using those features of Ruby, it’s relatively likely that you are depending on some third party library that, under the hood in a way that you don’t appreciate, is probably using some of that stuff. And so to flush all of that dynamic functionality out of the language would probably hobble it enough that people would say, “If I’m going to program in a language that has these restrictions, there are lots of better languages”. Once you take that bit out of Ruby you probably lose enough of the magic of Ruby that what you’ve ended up with is sort of not making anyone happy I would guess.

Mark: Yes, I mean there's some very clever things in Ruby on Rails for example. I remember when I looked at it, you can actually call a procedure that for example, or a method, that doesn't actually exist and it does something; it actually parses the name and can...

Yes that's right; method_missing.


7. How does that work?

It’s just, like many other dynamic languages, it gives you a hook for where you are able to catch the situation when someone has called a method that's undefined. There is a method called “method_ missing” in Ruby and if you define method_missing on a class, then any message that gets sent into an instance of that class where there is no corresponding method definition, you can just do whatever you like; and perhaps what you'll do is you’ll dynamically define a method that does by looking at the name of that method... And a lot of the testing infrastructure in Ruby takes advantage of this. It's one of the reasons why writing tests and doing TDD and BDD in Ruby is easier than it is in languages like Java, is that you can very easily build these domain specific languages in Ruby that allow you to sort of call arbitrary methods whose names follow a certain pattern, and then when you try to call that method, it will get dynamically created for you. So it makes it very easy to do this kind of convention over configuration type style of development, where you are spending less time setting up what you want to happen and more time just kind of saying what you want to happen and, if you are lucky, the system will respond in the correct way.

Mark: Tom thanks a lot for that; it’s been a pleasure talking to you and if anyone wants to see Tom’s talk they can find it on the website here; and other than that we'll say goodbye now!

Thanks very much!

Jun 07, 2014

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p