InfoQ

News

Aster In-Database MapReduce

Posted by R.J. Lorimer on Sep 21, 2008

Community
Java
Topics
Platforms ,
Enterprise Architecture ,
Database Design ,
Cloud Computing ,
Fault Tolerance
Tags
MapReduce ,
Database
Aster Data Systems recently announced Aster In-Database MapReduce, a component of their nCluster database.

MapReduce has been discussed at length at InfoQ, and is a programming model originally introduced by engineers at Google as a scalable approach to processing large data-sets.

The nCluster database is labeled by Aster as a massively parallel processing (MPP) database. The parallel architecture of nCluster is described on their website in this way:
Aster nCluster is built on a unique, multi-tiered nCluster architecture which consists of three separate classes of nodes: Queens, Workers, and Loaders. The three-tier design encapsulates a clean separation of roles for analytic processing. Each tier can be independently and incrementally scaled in response to the workload characteristics – adding more capacity (Workers), loading bandwidth (Loaders), or concurrency (Queens) on an as-needed basis.
The MapReduce implementation provided in Aster nCluster allows for the execution of MapReduce calculations within the database, using this same architecture:
Just like its massively parallel execution environment for standard SQL queries, Aster nCluster now adds the ability to implement flexible MapReduce functions for parallel data analysis and transformation inside the database. Aster nCluster In-Database MapReduce functions are simple to write and are seamlessly integrated within SQL statements. They rely on SQL queries to manipulate the underlying data and provide input. The functions can procedurally manipulate such input data and provide outputs that can be further consumed by SQL queries or be written into tables within the database.
SQL/MR is a special SQL MapReduce function library introduced by Aster that can be used to invoke map-reduce algorithms within the nCluster platform. Aster supports polymorphic functions and dynamic typing, and MapReduce calculations may be developed in languages such as Java, Python, C++ and others.

More information about In-Database Map Reduce and the nCluster database is available on the Aster Data Systems website.

No comments

Watch Thread Reply

Educational Content

Rails in the Large: How Agility Allows Us to Build One Of the World's Biggest Rails Apps

Neal Ford shows what ThoughtWorks learned from scaling Rails development: infrastructure, testing, messaging, optimization, performance.

Stuart Halloway on Clojure and Functional Programming

Stuart Halloway discusses Clojure and functional programing on the JVM in depth, and touches on the uses of a number of other modern JVM languages including JRuby, Groovy, Scala and Haskell.

Oren Teich and Blake Mizerany on Heroku

Oren Teich and Blake Mizerany talk about the technology behind Heroku and the benefits of the new add-on system.

Security for the Services World

Chris Riley presents security issues threatening service based systems, examining security threats, presenting measures to reduce the risks, and mentioning available security frameworks.

Navigating The Rapids:Real-World Lessons in Adopting Agile

This talk investigates technical issues encountered when moving to an Agile process.

Codename "M": Language, Data, and Modeling, Oh My!

Don Box and Amanda Laucher present “M”, a declarative language for building data models, domain models or external DSLs. Don Box's demos show some of M’s features and latest changes of the language.

SOA Manifesto - 4 Months After

It is four months since the SOA manifesto was announced; InfoQ interviewed the original author’s to get insight into the motivations and the process behind the initiative.

Memory Barriers and JVM Concurrency

This article explains the impact memory barriers, or fences, have on the determinism of multi-threaded programs.