Bio Justin Sheehy is the CTO of Basho Technologies, the company behind the creation of Webmachine and Riak. Damien Katz is the creator of CouchDB and founder of CouchOne.
The Erlang Factory is an event that focuses on Erlang - the computer language that was designed to support distributed, fault-tolerant, soft-realtime applications with requirements for high availability and high concurrency. The main part of the Factory is the conference - a two-day collection of focused subject tracks with an enormous opportunity to meet the best minds in Erlang and network with experts in all its uses and applications.
Damien Katz: I am Damien Katz, I am the creator of Couch DB and the CTO of Couch Base.
Justin Sheehy: I am Justin Sheehy, I am the CTO of Basho Technologies and one of the many things we are known for is Riak, which we will talk about here today, Webmachine and other things like that.
2. Riak and Couch DB. Some people find similarities inside, some people can distinguish between them but there seem to be quite a difference between them, can you tell us these differences between the niche of each of these data source.
Justin Sheehy: Sure, I think that one of the things with the whole NoSQL categorization of things that confuses people is that you’ve got all these different systems that are really - might solve overlapping needs, but they really have different sweet spots and different foci and because they are all in this category you go "Well which of these NoSQLs should I use?" Somebody asking that question probably doesn’t need any of these things, and I think the distinction is actually pretty significant technically where the strengths are even though we have some valuable things in common. Now Damian will talk about Couch but with Riak our initial goals were really to solve an availability problem and it was really about using distribution and building a distributed system that we would understand at the distributed system level, you know what the failure characteristics were and exactly what ways you had to manage a system to know when you will be able to stay write available and so on.
And there is lots and lots of things that make up Riak, there is horizontal scalability and all the sorts of things you have in a storage engine but if you start from that assumption, from what the first problem was, it shapes the architecture of the whole system. I think while we are overlapping some layers I think the initial problems being solved when Damian started Couch DB was not the exact same as we were working on.
Damien Katz: Exactly. There is definitely clearly an overlap but there is overlap with relational technologies as well, we are database technologies so there is clearly going to be big overlaps in terms of storing something and getting it back. The use cases and the sweet spots are where the big differences lie. With Couch DB it was about document databases and what the interesting things that you can do with those. So one of the killer features that Couch DB focuses on is replication and offline use cases. And not necessarily offline but rather disconnected so typically we think of offline is "I am offline I have my laptop but I have no network access".
So that’s one use case where you have a Couch DB database on your laptop all your information, you can query, update it, or whatever and then when you get back the network, you can synchronize that. But then there is also cross data center, so you have your New York data center, your Europe data center, your Asia data center, and those synchronize and the connection between those can get cut or whatever and you still need those to be able to operate independently. And so the synchronization and the being able to operate independent replicas of the database was really the focus that I was going after. And also the relational at the time I started Couch DB when people said databases they really meant relational databases.
There was no NoSQL movement at the time, and my motivation really was that there is a lot of applications out there that weren’t getting built because it was too difficult to build on top of relational database, it was just too much work, too expensive and so a lot of applications didn’t get built, instead people would just send around Excel spreadsheets and PowerPoint presentations, and Word documents and then merging all this stuff and doing it through email. And so that was also a big motivation of mine was just to make it easier to build a certain class of applications, document oriented applications, so it wasn’t so much about scale as it was just about making it easier for people to store data that previously was difficult.
Justin Sheehy: We do, so it’s a very different idea than the Couch DB replication, we focused much less on what I think is clearly one of the great sweet spots of Couch DB, the user takes their application, disconnects, does things, reconnects, and everything just works out, and with Riak there is two different sorts of replication you can think of: one of them is within a Riak cluster which typically means all machines that are in the same data center at the very least, there is some replication but it's not a whole database replication, it’s subsets of the database being spread across the cluster for availability purposes. So it’s not at that scale about whole database duplication as it is about redundancy throughout the system so you can think of that cluster as being a database in itself and that’s what we really mean by distributed system, right?
In that the database isn’t on a machine and then the database distributed on another machine, we have a cluster that itself runs a database with pieces of it distributed and replicated however much you choose to make it be, you can make trade offs between your physical resources and how much safety you want for failure. There is another layer of replication in that you can have a Riak cluster replicate to another Riak cluster, in the same datacenter in another data center or something like that, and that’s either for geographic locality or disaster recovery or so on, but that’s implemented in an entirely different way than in-cluster replication it’s really solving for a different class of problems.
Damien Katz: When you are evaluating the systems you really should look at what your use cases are, what the problem you are trying to solve is, is it scale, for instance it might not be scalability it might be that you have a rapidly evolving schema and so it’s not that you have mass amounts of data but you are constantly doing alter table statements that bring your server down too often. So you really have to look at what the exact problems are that you are trying to solve, and then try to figure out which of the relational or non-relational technologies are best going to fit that. It’s not necessarily about looking at all the technologies and figuring out what feature sets they have and then add them up and that’s the best one. You really do need to figure out what your problems are you are going to solve.
Justin Sheehy: And if you don’t know how all those different system work you do have to narrow it down, there is an enormous field, all of the databases going on right now, and so you don’t want to look at them all but you also don’t want to try and pick between two systems that couldn’t be more alike, or couldn’t be more different. So I think there are a few ways to start figuring out which systems might align with your use case. And probably a few good categories to start with might be to look at your data model that your application demands or just you don’t always everything about the way your data is shaped, what is the fundamental shape of the application you are building and the data it looks at and then some things fit a relational model almost exactly, some want very simple key/value models, some want certain sorts of batch processing, column storage things like that.
I’d say the second thing you usually want to care about and I don’t mean these in priority order, I mean there are a few things here, would be the distribution model and whether that’s solving a problem you need, whether that’s scale out or a failover of individual machines, and that varies widely across the system just like data model does, and not just Riak and Couch DB, this is across all databases. And one other thing I’d say you need to look at is the durability model and this is one of the things that you know with a database we’ve got used to all of these three choices being mostly the same for more traditional databases like MySQL, PostgreSQL , Oracle, and so on, and your durability model matters a lot too.
And it’s what guarantees you have about data you’ve put in this thing and what you are getting out of any potential compromises there, both Riak and Couch DB for instance, Riak by default - you can swap it out - and Couch DB have an append only, very strict approach to everything is durable all the time, there are some systems that have got other advantages, by trading away some of the durability guarantees. How much you care about that is really business specific.
Damien Katz: Right, yes, one other thing is that some systems tend to… they often don’t talk about some of their disadvantages much and durability might be one of those disadvantages that people aren’t aware of until it bites them at the wrong time and then they lost data. There are a lot of things to definitely be aware of when you are picking these databases.
Justin Sheehy: Sometimes that’s even the right tradeoff to make, the danger is that you don’t know you’re making it, in all of these cases. It doesn’t matter which element of the system you are talking about, you don’t need to pick the best, in some of these cases there isn’t a best model but you need to be informed about what you care about. And that might help you narrow down which of all the database system out there could help you solve your problems.
Justin Sheehy: After C++.
Damien Katz: The first versions were in C++, initially I chose C++ because I have written a ton of code I was very familiar with C++ and that’s what I felt most comfortable in. Then I started getting into the concurrency aspects of the database and I was going to go into the traditional threading and locks and all that. That’s when I started getting serious about investigating Erlang and spent a good solid week really digging into it and it clicked and when it did, that’s when I realized I could write the whole of the database in Erlang and save myself a whole lot of time. And I threw away about six months worth of work, C++ work and rewrote all that in Erlang in about a little over a month, and actually had a more functional project at the end of it. And this was a language that I had learnt in a week.
And so it was just the astounding productivity gain, that I got from it was amazing, and it wasn’t just the productivity gains that I was looking for, I was looking for reliability and concurrency that were promised, the fact that it was so strongly used in telecom, and the reasons that it was used in telecom the concurrency and the reliability and the failover were the same reasons I thought it would be great for a database and it turned out to be true. So yes, it’s been a great choice.
Justin Sheehy: So we had some of the same reasons, and just like I said earlier that one of our initial core goals with Riak was to build a highly available and predictaby available system going with a platform that has the kind of concurrency primitives built in for building reliable systems, writing something in Erlang doesn’t make it reliable, but especially if you use all the OTP conventions and applications and supervisory model and so on it’s very straight forward, not easy but straight forward to build something that is really hard to break and so you can blow up a part of Riak if you are mean enough to it and not only will the rest of the system keep going, but that part will be cleanly restarted and keep running.
That was sort of our core original reason, we’ve actually found a large number of other reasons over the past few years as well. The building blocks for doing things like state machines and so on that you care about both in the durability part of the database and in the network connection part are there to a level of quality that no other language and platform has. And because it’s a system with garbage collection that’s really made for soft real time systems, and it’s per-process garbage collection, we were able to build systems with very flat latency curves that just don’t spike up.
Damien Katz: Unlike Java.
Justin Sheehy: Right, and the JVM is a fantastic piece of technology that you can do a lot with but it’s because it wasn’t designed for systems with soft real time guarantees and it doesn’t have this notion of lightweight independent heaps. It’s very hard to get a Java program without an enormous amount of tuning that just doesn’t ever spike to the GC and with Erlang it’s the norm.
Damien Katz: I agree with that, there is not many garbage collected server side languages that do what Erlang does, and Erlang just absolutely nails it in terms of you need a server side language that is just going to run and run and run and run and run and not have these weird pauses, and not lock up, and at two AM you are going to get a call "Why has the server locked up again".
Justin Sheehy: And if things do go wrong too, you’ve got this fantastic ability to introspect a running Erlang system to a degree that if you have it on something else you have to build it yourself. With an Erlang node, especially when you set up to be prepared for this if you do have things go wrong, whether it is your software or the hardware on the machine, or anything, you can do a remarkable amount of live debugging without having much impact on the running system.
Damien Katz: Yes. And so Erlang is absolutely brilliant in that regard.
Justin Sheehy: I wouldn’t say the same across the board, there are pieces of what Erlang does well are present or appear in other system. You got the Akka system for Scala, that are rebuilding a number of the pieces for OTP and you have a concurrency model in Clojure for instance that is only about local concurrency not about remote concurrency and it’s not the same as Erlang’s but it’s like Erlang and it’s only those two languages in terms of even remotely close to mainstream have a concurrency model that is pretty straight forward to reason about and you don’t have to just guess what is going to happen when you are running multi-threaded. It’s typically very dirty and difficult, to have a highly concurrent program that you know exactly how it’s going to behave under load, and so people are working on these problems, but there is no other system out there that brings the set of things to the table that Erlang does for us that we really like.
Damien Katz: I don’t know of a language that does but the closest thing that I’ve seen in terms of a production environment is basically Apache server with PHP and sub-processes, kind of the Unix philosophies of process separation it’s kind of the grand daddy of the Erlang philosophy. So Erlang kind of takes the Unix philosophy and encapsulates that into a language and sort of pushes it to the extremes. But you see that already kind of existing inside of Unix you have processes and you can spawn sub-processes and then you have the isolation and then each process has its own memory heap and things like that and then with Apache web server it spawns off child process like PHP and those are isolated.
For instance, the Apache PHP web server even if that’s programmed by somebody who doesn’t know what they are doing they are frequently very hard to kill because even if it crashes or something like that it’s protected and the next request that comes in it will spawn off a new one. So it’s kind of the philosophy that is there in Unix is kind of inherited in Erlang and strengthened and taken to an extreme. I’ve always kind of seen that kind of lineage there and that’s the only other place that I’ve really seen it and Unix is this thing that existed now for thirty plus years and isn’t going away and is one of the few models that really have survived.
Justin Sheehy: The main difference on that model is that with Erlang OTP you have a system that allows you to provide already the plumbing between those independent isolated pieces. And that’s really one of the extra pieces of the value you get. If you want to have a state machine between communicating independent processes you don’t have to make that work, right. You just want to add your own logic and if you want to have one of those processes your server spawned do something when another one does get killed by some bad code or some bad user. You don’t have to write all the logic to make that work, it’s all of the interrelationships between - I know that sounds paradoxical – but the interrelationships between these independent parts that really gives you all that power with Erlang and then you have all the independence you want, but the ability to easily write a system that expresses and takes advantage...
Damien Katz: It monitors it.
Justin Sheehy: Definitely. I think it’s kind of crazy to be in the business either one of us and not be looking for inspiration where it appears. And some of the pieces of Couch DB don’t fit at all the way we are doing things and other parts we see these guys make a change and we say "Oh, that’s really paying off for these users, we should look more into that and see what it is there".
Damien Katz: Yes absolutely a lot of the full text stuff that you guys have been doing I am keeping a close eye on that because we definitely want to use some of that as well, they did some really cool stuff.