Oozie Eclipse plugin is a new tool for editing Apache Oozie workflows graphically inside Eclipse. Usage of this plugin allows to skip hard to develop and maintain process definition in HPDL. Instead a process graph is defined graphically by placing process actions on pallet and connecting them. An article introduces Eclipse Oozie plugin and provides an example of its usage.
Many of the Big Data technologies in common use originated from Google and have become popular open source platforms, but now Google is bringing an increasing range of big data services to market as part of its Google Cloud Platform. InfoQ caught up with Google's William Vambenepe, who's lead product manager for Big Data services to ask him about the shift towards service based consumption.
As data grows exponentially, the modern Hadoop ecosystem provides not only a reliable distributed aggregation system that delivers data parallelism, but also analytics for great data insights. In this article Monica Beckwith, starting from core Hadoop components, investigates the design of a highly available, fault tolerant Hadoop cluster, adding security and data-level isolation.
The new “Hadoop in Practice. 2 Edition" book by Alex Holmes covers a lot of topics building Hadoop code and organizing data to support code simplicity and execution speed.
Datameer, a big data analytics application for Hadoop, introduced Datameer 5.0 with Smart Execution to enhance the data analytics. InfoQ spoke with Matt Schumpert from Datameer about the new product.
This article discusses what stream processing is, how it fits into a big data architecture with Hadoop and a data warehouse (DWH), and what technologies and products you can choose from. 8
GridGain announced In-Memory Accelerator for Hadoop, offering benefits of in-memory computing to Hadoop applications. InfoQ spoke with Nikita Ivanov from GridGain about the product's architecture.
InfoQ spoke with Rich Reimer, VP of Marketing and Product Management at Splice Machine about the architecture and data patterns for SQL-on-Hadoop technologies.
Lambda Architecture proposes a simpler, elegant paradigm designed to process large amounts of data. In this article, author discusses Lambda Architecture with the help of a sample Java application. 20
In this article, authors discuss the role of big data and Hadoop in security analytics space and how to use MapReduce to process data for security analysis.
How to use various tools such as Apache Avro, Apache Crunch, Cloudera ML and the Cloudera Development Kit to build applications that use Hadoop.
Jon Natkins explains in this article how to create a personalized recommendation system fed with large amounts of real-time data using Kiji, which leverages HBase, Avro, Map-Reduce and Scalding.