HPCC Systems, which is part of LexisNexis, is launching this week its Thor Data Refinery Cluster on the Amazon EC2. HPCC Systems is an enterprise-grade, open source Big Data analytics technology platform capable of ingesting vast amounts of data, transforming, linking and indexing that data, with parallel processing power spread across the nodes.
eBay presented a keynote at Hadoop World, describing the architecture of its completely rebuilt search engine, Cassini, slated to go live in 2012. It indexes all the content and user metadata to produce better rankings and refreshes indexes hourly. It is built using Hadoop for hourly index updates and HBase to provide random access to item information.
Hortonworks, a company created in June 2011 by Yahoo! and Benchmark Capital, has announced the Technical Preview Program of Data Platform based on Hadoop. The company employs many of the core Hadoop contributors and intends to provide support and training.
A new post by Joe McKendrick outlines Hadoop’s ability to significantly simplify enterprise SOA implementation through improved data access services build on a common enterprise data platform.
Companies rely more and more on big data when making their decisions. Amazon, Cloudera, and IBM have announced their Hadoop-as-a-Service offerings, while Microsoft promises to do the same next year.
Microsoft announced that the next version of SQL Server, known by the codename "Denali", will be called SQL Server 2012. It will feature the big data capabilities of Apache Hadoop and Power View, a touch-based business intelligence tool.
Twitter has open-sourced Storm, its distributed, fault-tolerant, real-time computation system, at GitHub under the Eclipse Public License 1.0. Storm is the real-time processing system developed by BackType, which is now under the Twitter umbrella.
MapR Technologies released a big data toolkit, based on Apache Hadoop with their own distributed storage alternative to HDFS. The software is commercial, with both a free edition, M3, as well as a paid edition, M5. M5 includes snapshots and mirroring for data, Job Tracker recovery, and commercial support. MapR's M5 edition will form the basis of EMC Greenplum's upcoming HD Enterprise Edition.
Yahoo spun-out its core Hadoop team, forming a new company Hortonworks. CEO Eric Baldeschwieler presented their vision of easing adoption of Hadoop and making core engineering improvements for availability, performance, and manageability. Hortonworks will sell support, training, and certification, primarily indirects through partners.
VMware vFabric SQLFire is an in-memory distributed SQL-based cache which can work with a traditional database to persist data to disk.
DataStax described Brisk their new Hadoop distribution that stores data in Cassandra, EMC published an ad that promised big news about Hadoop and Greenplum on May 9th, and GigaOm claimed that MapR Technologies is building a proprietary version of Hadoop. DataStax told InfoQ there are production Cassandra clusters of 700 nodes, storing hundreds of terbaytes, and doing 200,000 writes per second.
Yahoo recently announced and presented a redesign of the core map-reduce architecture for Hadoop to allow for easier upgrades, larger clusters, fast recovery, and to support programming paradigms in addition to Map-Reduce. The new design is quite similar to the open source Mesos cluster management project - both Yahoo and Mesos commented on the differences and opportunities.
JasperSoft announces reporting support for Hadoop and leading NoSQL databases.
Membase and Cloudera announced integration of the Membase NoSQL database and Cloudera's Distribution for Hadoop, the distributed map-reduce and storage system, allowing for bi-direction data replication between the systems.
Cloudera recently announced Cloudera Enterprise, a commercial bundling of Hadoop and a dozen other supporting open source projects. InfoQ interviewed Product Manager Charles Zedlewski for more detail about what this means for conventional enterprises and the future face of Hadoop.