CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

The team behind CockroachDB, an open source datastore project, has recently announced its initial alpha version. Inspired by Google’s Spanner project, CockroachDB aims to address the current lack of an open source, scalable, geo-replicated, ACID compliant database.

Traditionally, NoSQL datastores have supported multi datacenter configurations through the use of asynchronous replication between datacenters. This is the recommended approach in MongoDB and it is the only approach available in Riak, Cassandra and HBase. However, asynchronous replication prevents consistency guarantees between datacenters. Moreover, most of the first generation NoSQL datastores failed to support arbitrary cross key transactions. Recently, new systems such as FoundationDB and HyperDex have been designed with cross key transaction support. However, these systems caution against multi datacenter configurations.

Against that backdrop Google announced their Spanner system in 2012 which broke new ground supporting both transactions and multi datacenter consistency. While the Spanner paper inspired many, it wasn’t until the CockroachDB team began on their implementation that an open source project tried to replicate Spanner’s challenging mix of features.

In the draft specification, Spencer Kimball, who worked on the Colossus file system at Google, describes CockroachDB as supporting

ACID transactional semantics and versioned values as first-class features. The primary design goal is global consistency and survivability, hence the name. … Cockroach implements a single, monolithic sorted map from key to value where both keys and values are byte strings.

CockroachDB’s transactional features are provided in two ways. Kimball explains that

If all keys affected by a logical mutation fall within the same range, atomicity and consistency are guaranteed by Raft; this is the fast commit path. Otherwise, a non-locking distributed commit protocol is employed between affected ranges. Cockroach provides snapshot isolation (SI) and serializable snapshot isolation (SSI) semantics … SSI is the default isolation; clients must consciously decide to trade correctness for performance.

The non-locking distributed commit protocol is still a work in progress but Kimball references papers from The University of Sidney, Yale, and draws special attention to a paper from Yahoo! as potential options for implementing SSI semantics.

The current alpha version provides a very minimal subset of the desired functionality. Cluster initialization and joining, gossip network, and a basic key-value REST API are currently supported but there is no raft consensus, range splitting or transactions. The project is written in Go and they are currently looking for contributors.