MongoDB 1.6 Adds Sharding and Replica Sets

Posted by Abel Avram on Aug 11, 2010

Sections

Operations & Infrastructure,

Architecture & Design

Topics

NoSQL ,

Architecture ,

Performance & Scalability ,

Database

Share |

RELATED Research

MongoDB 1.6 is a major release addressing the scaling-out issue through sharding and adding replica sets for automatic failover and recovery.

Sharding

The main improvement coming with MongoDB 1.6 is the ability to scale-out through sharding. MongoDB can automatically distribute databases, collections or objects in a collection over multiple shards without any downtime. Sharding collection is preferable because in many cases some data collections tend to be much larger than others, so it would be unprofitable to distribute the entire database.

A sharded configuration contains shards, routing processes and configurations servers, as shown in the following picture:

mongod – is the main database process, representing one shard and being organized in replica sets in order to provide automated failover. One of the mongod processes is the master in its replica set. If the master goes down, another server is delegated as master.

mongos – is a routing process, making the interaction of the client with the sharded database as connecting to a single server. There can be as many mongos servers as necessary, they having no shared state.

config servers – Each configuration server contain the cluster’s metadata, what shards exist in the system, and what chunks of data is on each shard. There are multiple configuration servers for protection. If one of them goes down, the configuration servers enter in read-only mode, but the shards continue to run in read/write mode.

Replica Sets

Another major feature introduced in MongoDB 1.6 is Replica Sets, a replication feature based on the initial master/slave replication but adding automatic failover and recovery. There can be up to 7 servers in a cluster, and any node can be the primary node. All writes are directed to the primary node, from where they are replicated while reads can be performed on any node.

Other Improvements

$or queries – The $or operator supports boolean or expressions in queries:

db.foo.find( { name : "bob" , $or : [ { a : 1 } , { b : 2 } ] } )

Windows Service – mongod runs on Windows as a service now, but not mongos.

w option – Writes can be blocked until the operation propagates to n servers in the set.

According to Dwight Merriman, CEO and co-founder of 10gen, the company providing support for the document database, MongoDB 1.6 is already used in production by bit.ly and foursquare. According to Merriman, bit.ly has about 50M users with 10K using the servers concurrently during peak times. Foursquare has millions of users and it migrated to sharded MongoDB from Postgres, using geospatial indexing.

The Release Notes and MongoDB Jira contain more information on what’s new in 1.6.

RelatedVendorContent

Troubleshoot Java/.NET performance while getting full visibility in production

RDBMS to NoSQL: Managing the Transition

Banking Case Study: Scaling with Low Latency using NewSQL

Good Relationships: The Spring Data Neo4J Guide Book

NOSQL for the Enterprise

Related Sponsor

Neo4j is a robust, high-performance, scalable graph database. It is the only NOSQL database that solves the complex, connected data challenges that enterprises face today.