Pivotal has decided to open source core components of their Big Data Suite and has announced the Open Data Platform, an initiative promoting open source and standardization for Big Data.
Pivotal came later to the Big Data market, after some of the earlier players such as HortonWorks, Cloudera and MapR. But now, to address “fragmentation and vendor lock-in” in the big data space, Pivotal has decided to open source a number of products from its Big Data Suite, namely Greenplum Database - parallel processing data warehouse –, HAWQ – an ANSI-compliant SQL on Hadoop query engine, and GemFire – a distributed in-memory NoSQL database.
Michael Cucchi, Sr. Director of Product Marketing at Pivotal, provided more details on this process and the reasons behind it. According to Cucchi, while customers liked “the Pivotal Big Data Suite’s flexibility”, their buying preferences changed because “customers these days are seeking to incorporate open source technology as much as possible, especially in the area of IT infrastructure.” As a result, Pivotal decided to open source the core of their Big Data Suite, and Cucchi mentioned their intent to open source all components of the suite.
According to Cucchi, the open sourcing process has been “in the works for some time” due to the complexity related to licensing, intellectual property and product governance, providing some details for what’s coming next:
Our detailed plans are still being finalized, but we plan to begin release and incubation of Pivotal GemFire,Pivotal HAWQ, and Pivotal Greenplum Database in a quarterly cadence. We’re closing in now on the structure of ownership of GemFire, Greenplum Database, and HAWQ code to the most appropriate entity for working with the big data community. As we get closer to actual release milestones, we will continue to update on the progress of each product.
Pivotal has also announced the Open Data Platform (ODP), an initiative of 15 companies including Hortonworks, IBM, Infosys, GE, SAS, that are promoting open source and standardization in the Big Data space. The first components that will be taken care of by ODP are Ambari, HDFS, MapReduce, and YARN, and it is quite likely that Pivotal will entrust the source code of their suite to ODP.
In the same time, the Big Data Suite has been enhanced with a number of services: the ability to deploy the suite on Cloud Foundry taking advantage of the Operations Manager, and integration with Spring XD, Redis, and RabbitMQ. The next version of Pivotal HD will integrate with Spark and “all of the available Apache projects”, according to Cucchi.