ActiveWarehouse, a New Step for Enterprise Ruby
Ruby and Rails continue to generate a lot of buzz in the overall software development community, but debate rages over how it will accomodate enterprise needs. For instance, is Rails able to handle large amounts of business data? With the new release of ActiveWarehouse, open-source Rails programmer Anthony Eden delivers a plugin that makes it easier to effectively build data warehouses using Ruby on Rails. Right now, ActiveWarehouse is one of the most active RubyForge projects, and progress along the features roadmap looks impressive.
The ActiveWarehouse plugin simplifies building data warehouses in Rails. A data warehouse is a database designed specifically for analytical use as opposed to operational transaction processing. Typically a data warehouse houses data which spans several years and is sourced from numerous operational databases. Data warehouses are usually highly de-normalized which is contrary to transactional systems which tend to be normalized. The links in the side bar provide additional information.
In this release you'll find notably : generators for facts, dimensions and cubes; Multi-dimension support; Automatic creation of aggregates and many other features, with a full pipeline of planned enhancements on the way.
To get the data from multiple data sources into the data warehouse, ActiveWarehouse is paired with the ActiveWarehouse ETL component.
The ETL handles most of the basic source types which would be used when integrating a fairly current system (delimited, fixed-width, XML and database sources). It can also be extended with custom parsers, so that part is handled at the moment. There are also enough transforms to be useful and adding new ones is easy. The system is definitely extensible.
Other functionalities also available are notably : Virtual source fields; Support for pre and post processing code; ETL Domain Specific Language (DSL) control files. Bulk loading is available only for MySQL at the moment. Anthony is also working on performance issues, which are always a crucial in this domain.
ActiveWarehouse and ETL component abilities are demonstrated through a comprehensive tutorial.
Any plans on supporting MDX?
Re: Looks Good
Whatever your connotation of "enterprise" is, what it boils down to is how much your VM can take under load. The JVM can, Erlang too and there are others.
The Ruby VM however is lagging many years and millions of dollar/euro behind. Don't take my word for it, ask the people that work on it.
You are right, the performance of the Ruby interpreter *is* an issue in certain parts of ActiveWarehouse, specifically data aggregation for large data sets. I remember the same was true back in the early days of Java as well, so I have confidence that the performance of Ruby will continue to improve. One day we will have both performance and joy of development at the same time. ;-)
Tom Gilb & Kai Gilb Jan 26, 2015