BT

ActiveWarehouse, a New Step for Enterprise Ruby

by Sebastien Auvray on Mar 28, 2007 |

Ruby and Rails continue to generate a lot of buzz in the overall software development community, but debate rages over how it will accomodate enterprise needs. For instance, is Rails able to handle large amounts of business data? With the new release of ActiveWarehouse, open-source Rails programmer Anthony Eden delivers a plugin that makes it easier to effectively build data warehouses using Ruby on Rails. Right now, ActiveWarehouse is one of the most active RubyForge projects, and progress along the features roadmap looks impressive.

The ActiveWarehouse plugin simplifies building data warehouses in Rails. A data warehouse is a database designed specifically for analytical use as opposed to operational transaction processing. Typically a data warehouse houses data which spans several years and is sourced from numerous operational databases. Data warehouses are usually highly de-normalized which is contrary to transactional systems which tend to be normalized. The links in the side bar provide additional information.

In this release you'll find notably : generators for facts, dimensions and cubes; Multi-dimension support; Automatic creation of aggregates and many other features, with a full pipeline of planned enhancements on the way.

To get the data from multiple data sources into the data warehouse, ActiveWarehouse is paired with the ActiveWarehouse ETL component.

The ETL handles most of the basic source types which would be used when integrating a fairly current system (delimited, fixed-width, XML and database sources). It can also be extended with custom parsers, so that part is handled at the moment. There are also enough transforms to be useful and adding new ones is easy. The system is definitely extensible.

Other functionalities also available are notably : Virtual source fields; Support for pre and post processing code; ETL Domain Specific Language (DSL) control files. Bulk loading is available only for MySQL at the moment. Anthony is also working on performance issues, which are always a crucial in this domain.

ActiveWarehouse and ETL component abilities are demonstrated through a comprehensive tutorial.

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Looks Good by Frank Bolander

Good job Anthony. The ETL stuff looks very interesting.

Any plans on supporting MDX?

Re: Looks Good by Anthony Eden

No plans for MDX at the moment. ActiveWarehouse isn't a middleware app at the moment, rather it is designed to be used directly from Rails. That's not to say that it couldn't become a standalone middleware app in the future, it's just not something I or the other authors need at the moment. Feel free to jump in and scratch any itches that you have, though. :-)

Enterprise by Steven Devijver

If data warehouse code is considered "enterprise" then many of the scripts I wrote in my Perl days are "enterprise" too.

Whatever your connotation of "enterprise" is, what it boils down to is how much your VM can take under load. The JVM can, Erlang too and there are others.

The Ruby VM however is lagging many years and millions of dollar/euro behind. Don't take my word for it, ask the people that work on it.

Re: Enterprise by Anthony Eden

Steven,

You are right, the performance of the Ruby interpreter *is* an issue in certain parts of ActiveWarehouse, specifically data aggregation for large data sets. I remember the same was true back in the early days of Java as well, so I have confidence that the performance of Ruby will continue to improve. One day we will have both performance and joy of development at the same time. ;-)

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

4 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT