InfoQ

InfoQ

Interview

My Bookmarks

Login or Register to enable bookmarks for unlimited time.

The content has been bookmarked!

There was an error bookmarking this content! Please retry.

Recorded at:
Recorded at

Nathaniel Talbott on Experiment Driven Design

Interview with Nathaniel Talbott by Obie Fernandez on Mar 19, 2010 Length 00:07:34     Download: MP3

Sections
Process & Practices,
Architecture & Design,
Development
Topics
Ruby ,
Delivering Value ,
Tools ,
Agile
Tags
TDD ,
BDD ,
RubyConf
 
Summary
Nathaniel Talbott discusses the concept of Experiment Driven Design.

Bio
Nathaniel Talbott is the owner of Terralien, co-founder of Spreedly, and Rubyist since 2001. He blogs at http://blog.talbott.ws/

About the conference
The International Ruby Conference or RubyConf, is the official annual gathering of Ruby developers from around the world, founded in 2001. RubyConf has historically been the venue for announcements regarding the future of the Ruby programming language as well as a place for the creators of all the major Ruby implementations to show their work, compare notes and involve the community.
I am here with Nathaniel Talbott who just wowed the Ruby community at RubyConf with a great new concept. Why don't you tell us all about it?
Yes, we talked about Experiment driven development, so a little bit of background about 9 months ago earlier this year, when I ran across this concept of Lean Startups, which is something that a guy named Eric Ries has been talking about a whole lot and he has basically taken the principles of Agile which we are all familiar with and put them together with these concepts from what's called "customer development" from a guy named Steve Blank, excellent book that everyone should read called "Four steps to Epiphany", just fantastic, it's like unedited Cafepress course notes from Steve Blank's course, the content is absolute gold.

And so I am reading through this stuff and I am learning about it and one of the things that Eric does when he is going through and talking about lean startups is he talks about like you should be a part of a process, part of what you need to do when you are starting a business is you need to be able to scientifically measure like instead of being opinions and conjectures around the table and just like my ego is bigger than yours in terms of what we should build and where the business should go; that development in particular can contribute to things by allowing the business to run experiments to say: "OK, you think that we need to collect all 5 of these fields, but I think we are going to double our signups if we only collect 2 fields."
Eric said they won't necessarily think of on their own, right?
Yes, absolutely. One of the ideas in the talk is that we need to start sort of reforming the conversation between the business side and the development side in terms of conversing around hypotheses. Instead of being business throwing features over the wall at development, instead the business needs to say: "Ok, here is what we think about what people are going to do.

Can you help us test that and figure out what people will do?" and so you can actually start to have fact based discussions around actual data you've collected, instead of being stuck in these never ending arguments about like: "Well, I read Jacob Nielsen said this thing one time and stop telling me this!" and the business going: "No, we have to collect this information or the business will collapse".
So how does the subject of developer driven experiments intersect with Ruby and RubyConf, why here, why now?
Why now is basically because I've gotten all fired up about this, just because as I went out and I was like: "Ok, Eric says I should run experiments" and then I am like: "How do I do it?" and you start looking at it and like Eric says "You should be able to write an experiment in one line of code" and I am like: "That sounds fantastic!" and then I go to actually try to do it and then you find out that there is this all this moving pieces to an experiment and it's fantastic that at the companies he's been at they have spent all this time building these internal frameworks, sort of allow them to write experiments in one line of code, but it doesn't help the rest of us.

And so it was just so frustrating to be like: "Ok, we really need to be doing this, but it's really hard and difficult" and when I looked back at the history of test driven development from a developer perspective and what it was like before we had frameworks like JUnit and TestUnit and SUnit, even SmalltalkUnit before that and NUnit and now we have like a whole plethora of different frameworks within the Ruby community, RSpec, etc.

It wasn't just a matter of us realizing that we should be writing test first that sort of caused the adoption of TDD. It was also the fact that we started to have these frameworks so you'd just be crazy not to do test driven development. It's like: "Why are we not doing this?" It's so easy to do. You go into a Rails project and you generate anything, and you generate stubs and you go and you run rake what's the default? It runs the test.

And so this infrastructure is really what helped get the momentum behind test driven development where it's really standard operating procedures especially in the Ruby community now. So what I want to see is the Ruby community just the way that we have this testing culture I want to see it having experimentation culture where the developer culture is constantly pushing back on the business going: "Do you have an experiment for that?" just the same way we say to each other: "Do you have a test for that?". So "Do you have an experiment for that? Why are you throwing an opinion at me? What's your hypothesis here and let me help you test it?

You say that this feature that is taking so much time is going to be really used, how many people are really using that? Let's actually test the usage. Oh, it turns out that you and one other user happen to use that feature. We are dropping it, it's ridiculous for us to keep spending money on this!" and having those kinds of conversations and make development an integral part of growing the business as opposed to just the separate part that kind of goes off and does their own thing and has tests that help them out, but it doesn't really feed back into the business.
How are you applying these concepts in your own business?
That is something that I am working on. I am really excited about A who does the Labnotes blog, I gave a preview of this talk a few weeks back and the day after I was at lunch with him and I was telling him about it and he got all fired up and put together a framework called Vanity, so that's at vanity.labnotes.org and I am really excited to take vanity now and put it in place because what I found is tried to apply this to my different businesses was that in client work it was like convincing a client that it was worth it to spend on the amount of infrastructure that we needed to produce to run the kind of experiments I thoughts would be valuable and sort of set up not just like one experiment, but sort of an ongoing culture of experimentation was prohibitive, like to take that to a client.

And the same was Spreedly as I looked and looked at doing in Spreedly it's like, well we are handling support requests we are trying to add these new features, how can we justify spending the time on experimentation even though we know it's really valuable it's really hard to justify. So it's something like Vanity now in place we can start and go in and it's getting to that point where it's like: "This is so easy. It's crazy not to." So I am really looking to start doing more and more and figuring out how to fit it into the businesses.
Final question, you brought Test Unit to the Ruby community years ago, right? And now you are bringing something that seems like you could be again like a land mark achievement. So where do you find inspiration to do these things?
Like I said a lot of the inspiration behind EDD specifically comes from Eric Ries and Steve Blank and what they are saying on the business side of things, but in general it's funny but a lot of my inspiration comes from trying to figure out what I should talk about at the next conference. Sometimes it like I want to go, I want to inspire people, I want to change behavior and so it's like what I am currently doing that I really feel like is a game changer and how can I bring that out in such a way as to really fire people up and get them change beahaviour.
And then this ends up giving you this something to work on?
Yes, absolutely.
show all  show all show all
  • This article is part of a featured topic series on Agile

No comments

Watch Thread Reply

Educational Content

Jesper Boeg on Priming Kanban

In this interview, Jesper Boeg, author of the new InfoQ book – Priming Kanban, discusses the keys to using Kanban effectively, and how to get started if you are currently using other approaches.

New-age Transactional Systems - Not Your Grandpa's OLTP

John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.

Cool Code

Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.

Collaboration: At the Extremities of Extreme

Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.

Yesod Web Framework

Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).

Transactions without Transactions

Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.

Attila Szegedi on JVM and GC Performance Tuning at Twitter

Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.

10 tips on how to prevent business value risk

One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.