Ron Bodkin of Big Data Analytics discusses early adoption of Hadoop, NoSQL and big data technologies. He discusses common patterns and explains how developers can write low-level primitives to optimize MapReduce function. Other topics include Hive, Pig, multi tenancy, and security.
In their article authors, Boris Lublinsky and Mike Segel, show how to leverage custom InputFormat class implementation to tighter control execution strategy of Maps in Hadoop Map Reduce jobs.
A new open source project – Dempsy adds one more option for people trying to do real time processing of big data. Comparable to Storm and S4 Dempsy is most applicable to near real time stream processing where latency is more important than guaranteed delivery.
After six years of gestation, Big data framework Apache Hadoop 1.0.0 was recently released. Core features in the release include Kerberos Authentication, support for Apache HBase and RESTful API to HDFS. InfoQ spoke with Arun Murthy, VP of Apache Hadoop, about the new release.
In traditional fashion, we celebrate the new year with a roundup of predictions in the SOA and Cloud space for 2012. This coming year the promising trends in big data and IT consumerization are expected to lead SOA and Cloud adoption. What is your prediction?
Who ever has wondered what kind of software is used by Santa Claus & Co, got a hint recently in youtube. This might irritate some software engineers who have assumed, Santa Claus would only use Open Source Software.
IBM has recently prototyped a software architecture that can deal with large amount of data flows. IBM’s software is built for the SKA telescope (Square Kilometre Array) and allows to automatically classify astronomical objects. Radio astronomer Melanie Johnston-Hollitt at Victoria University, Wellington , NZ, has collaborated with IBM for developing the system.
eBay presented a keynote at Hadoop World, describing the architecture of its completely rebuilt search engine, Cassini, slated to go live in 2012. It indexes all the content and user metadata to produce better rankings and refreshes indexes hourly. It is built using Hadoop for hourly index updates and HBase to provide random access to item information.
