While Relational Databases fit a client-server model, in a world of services new solutions are needed. RDBMS are subject to scalability issues: How to create redundancy, parallelism ?
Damien Katz, software engineer at MySQL introduced the four pillars of Data Management:
The features of RDDB at the moment are:
Firstly what lead you to start working on RDDB, at Rejectconf you were talking about a research project?
I consider RDDB to be a personal research project. Over the last year I've been heavily involved in analytical systems, developing data warehouses and the like. I've also been using Amazon's web services. RDDB will hopefully allow me to put the two together at some point so I can have an analytical database that runs on EC2 and S3. That's my primary goal and the driving factor behind the creation of RDDB.
In your daily job you're exposed to Data integration issues; Do you think Document Oriented Distributed Database are underused today and that they will be adopted more and more?
I'm not sure yet. There is a lot of history behind relational databases and they've had a lot of time to mature. On the one hand this makes them the obvious choice for operational systems since they can be trusted. On the other hand relational databases aren't necessarily the best choice for all types of data storage and lookup, so there are opportunities for new data stores, I'm just not sure of document oriented databases are going to be it - I think it will largely depend on their scalability and their ability to deal with massive amounts of documents without degradation of performance.
Is there still a place for RDBMS in a world of services model? While referential integrity, atomic updates and constraints makes sense in a client-server world, is it still relevant in a world of services?
It's still the standard by which others are judged, so I don't see relational databases going anywhere anytime soon. Ultimately I think we may be able to move past needing atomic updates if we have databases that are temporal in nature, thus removing the need for updates of any sort. Referential integrity can also probably go if we move to an environment where everything that is absolutely necessary is included in a resource and systems become more tolerant to missing links. Constraints will probably always be useful and perhaps they may become even richer with the ability to define logic for constraints.
How do you compare RDDB to CouchDB? (I know you're in early early stage of development, CouchDB is also). What would be the advantage of using RDDB over CouchDB Ruby binding?
I think I can answer these two together. Since CouchDB is written in Erlang and RDDB is written in Ruby, for a Ruby developer RDDB will be much more hackable. CouchDB uses the language features of Erlang for interprocess communication during distributed processing whereas Ruby relies on libraries, such as Rinda and the Ruby SQS library. For a Ruby developer the cost of getting RDDB up and running is going to be significantly less than CouchDB since all you have to do with RDDB is use RubyGems to install. The views in RDDB are written in Ruby whereas the CouchDB views are JSON (at least for now). I think that at the moment RDDB is more pluggable with different implementations for document store, view store and materialization store (all of which support RAM, file system, S3 storage). RDDB also has different implementations for materialization (such as local, Rinda and EC2), as well as threaded and non-threaded materializers.
We wrote an article about ActiveWarehouse a while back, how is the project going? Is it used in the enterprise?
ActiveWarehouse has been quiet lately. I believe that most of the work and usage is on the ETL side with the ActiveWarehouse ETL library. My goal is to release the 1.0 version of ActiveWarehouse ETL in the near future. As for the Rails plugin, it definitely needs more work on the display side before it can get raised up to a 1.0 version. Some people have expressed interest in revising the user interface code so we'll see where that leads.
[Relation Databases] become a single point of failure. In particular, replication is not trivial. To understand why, consider the problem of having two database servers that need to have identical data. Having both servers for reading and writing data makes it difficult to synchronize changes. Having one master server and another slave is bad too, because the master has to take all the heat when users are writing information.In addition, Assaf Arkin also believes that write consistency is the reason RDBMS are imploding under their own weight.
Features like referential integrity, constraints and atomic updates are really important in the client-server world, but irrelevant in a world of services.Those are typical issues that Document Oriented Distributed Databases are notably trying to address.
Damien Katz, software engineer at MySQL introduced the four pillars of Data Management:
- Save: Data saving should be secure (ie ACID), permanent and efficient.
- See: Data should be available for easy retrieval, integrate simple reporting methods and provide a (fulltext) search.
- Secure: Compartmentalization of data, allow ssl connection, assign users, groups and roles to data...
- Share: Be distributed, On and Offline.
Inspired by CouchDB and the notion that you insert documents into the database and then define views for querying, Anthony Eden started to write his own Document-Oriented Database: RDDB. An exhaustive review is already available.
What CouchDB is
What CouchDB is not
- A document database server, accessible via a RESTful JSON API.
- Ad-hoc and schema-free with a flat address space.
- Distributed, featuring robust, incremental replication with bi-directional conflict detection and management.
- Query-able and index-able, featuring a table oriented reporting engine that uses Javascript as a query language.
- A relational database.
- A replacement for relational databases.
- An object-oriented database. More specifically, CouchDB is not meant to function as a seamless persistence layer for an OO programming language.
The features of RDDB at the moment are:
InfoQ had the chance to catch up with Anthony and talk about RDDB, CouchDB and RDBMS.
- Documents are simply collections of name/value pairs.
- Views can be defined with Ruby code.
- A reduce block can be defined to reduce the initial mapped data from a view.
- Views can be materialized to improve query performance.
- Datastores/Viewstores/Materialization stores are pluggable. Current implementations are RAM, partitioned files/file system and Amazon S3.
- Distributed materialization may work, but it's going to be rewritten.
Firstly what lead you to start working on RDDB, at Rejectconf you were talking about a research project?
I consider RDDB to be a personal research project. Over the last year I've been heavily involved in analytical systems, developing data warehouses and the like. I've also been using Amazon's web services. RDDB will hopefully allow me to put the two together at some point so I can have an analytical database that runs on EC2 and S3. That's my primary goal and the driving factor behind the creation of RDDB.
In your daily job you're exposed to Data integration issues; Do you think Document Oriented Distributed Database are underused today and that they will be adopted more and more?
I'm not sure yet. There is a lot of history behind relational databases and they've had a lot of time to mature. On the one hand this makes them the obvious choice for operational systems since they can be trusted. On the other hand relational databases aren't necessarily the best choice for all types of data storage and lookup, so there are opportunities for new data stores, I'm just not sure of document oriented databases are going to be it - I think it will largely depend on their scalability and their ability to deal with massive amounts of documents without degradation of performance.
Is there still a place for RDBMS in a world of services model? While referential integrity, atomic updates and constraints makes sense in a client-server world, is it still relevant in a world of services?
It's still the standard by which others are judged, so I don't see relational databases going anywhere anytime soon. Ultimately I think we may be able to move past needing atomic updates if we have databases that are temporal in nature, thus removing the need for updates of any sort. Referential integrity can also probably go if we move to an environment where everything that is absolutely necessary is included in a resource and systems become more tolerant to missing links. Constraints will probably always be useful and perhaps they may become even richer with the ability to define logic for constraints.
How do you compare RDDB to CouchDB? (I know you're in early early stage of development, CouchDB is also). What would be the advantage of using RDDB over CouchDB Ruby binding?
I think I can answer these two together. Since CouchDB is written in Erlang and RDDB is written in Ruby, for a Ruby developer RDDB will be much more hackable. CouchDB uses the language features of Erlang for interprocess communication during distributed processing whereas Ruby relies on libraries, such as Rinda and the Ruby SQS library. For a Ruby developer the cost of getting RDDB up and running is going to be significantly less than CouchDB since all you have to do with RDDB is use RubyGems to install. The views in RDDB are written in Ruby whereas the CouchDB views are JSON (at least for now). I think that at the moment RDDB is more pluggable with different implementations for document store, view store and materialization store (all of which support RAM, file system, S3 storage). RDDB also has different implementations for materialization (such as local, Rinda and EC2), as well as threaded and non-threaded materializers.
We wrote an article about ActiveWarehouse a while back, how is the project going? Is it used in the enterprise?
ActiveWarehouse has been quiet lately. I believe that most of the work and usage is on the ETL side with the ActiveWarehouse ETL library. My goal is to release the 1.0 version of ActiveWarehouse ETL in the near future. As for the Rails plugin, it definitely needs more work on the display side before it can get raised up to a 1.0 version. Some people have expressed interest in revising the user interface code so we'll see where that leads.