InfoQ

InfoQ

News

My Bookmarks

Login or Register to enable bookmarks for unlimited time.

The content has been bookmarked!

There was an error bookmarking this content! Please retry.

Aster In-Database MapReduce

Posted by R.J. Lorimer on Sep 21, 2008

Sections
Operations & Infrastructure,
Enterprise Architecture,
Development,
Architecture & Design
Topics
Enterprise Architecture ,
Fault Tolerance ,
Platforms ,
Database Design ,
Java ,
Cloud Computing
Tags
MapReduce ,
Database
Aster Data Systems recently announced Aster In-Database MapReduce, a component of their nCluster database.

MapReduce has been discussed at length at InfoQ, and is a programming model originally introduced by engineers at Google as a scalable approach to processing large data-sets.

The nCluster database is labeled by Aster as a massively parallel processing (MPP) database. The parallel architecture of nCluster is described on their website in this way:
Aster nCluster is built on a unique, multi-tiered nCluster architecture which consists of three separate classes of nodes: Queens, Workers, and Loaders. The three-tier design encapsulates a clean separation of roles for analytic processing. Each tier can be independently and incrementally scaled in response to the workload characteristics – adding more capacity (Workers), loading bandwidth (Loaders), or concurrency (Queens) on an as-needed basis.
The MapReduce implementation provided in Aster nCluster allows for the execution of MapReduce calculations within the database, using this same architecture:
Just like its massively parallel execution environment for standard SQL queries, Aster nCluster now adds the ability to implement flexible MapReduce functions for parallel data analysis and transformation inside the database. Aster nCluster In-Database MapReduce functions are simple to write and are seamlessly integrated within SQL statements. They rely on SQL queries to manipulate the underlying data and provide input. The functions can procedurally manipulate such input data and provide outputs that can be further consumed by SQL queries or be written into tables within the database.
SQL/MR is a special SQL MapReduce function library introduced by Aster that can be used to invoke map-reduce algorithms within the nCluster platform. Aster supports polymorphic functions and dynamic typing, and MapReduce calculations may be developed in languages such as Java, Python, C++ and others.

More information about In-Database Map Reduce and the nCluster database is available on the Aster Data Systems website.

No comments

Watch Thread Reply

Educational Content

New-age Transactional Systems - Not Your Grandpa's OLTP

John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.

Cool Code

Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.

Collaboration: At the Extremities of Extreme

Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.

Yesod Web Framework

Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).

Transactions without Transactions

Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.

Attila Szegedi on JVM and GC Performance Tuning at Twitter

Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.

10 tips on how to prevent business value risk

One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.

Interview: Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives

InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.