Bulk data is commonly accessed via files & FTP. As the world moves toward APIs to facilitate collaboration, what are the requirements for data APIs? This article describes a meta-data driven architecture for bulk data ingestion. Two APIs operate in parallel to provide data changes as well as the data records themselves. An example demonstrates how API responses are parameterized using meta-data.
Synchronization of data across systems is expensive and impractical when running systems at scale. Traditional approaches for performing computations or information dissemination are not viable. In this article Basho Sr. Software Engineer Chris Meiklejohn explores the basic building blocks for crafting deterministic applications that guarantee convergence of data without synchronization.
Apache Spark is an open source big data processing framework built around speed, ease of use, and sophisticated analytics. In this article, Srini Penchikala talks about how Apache Spark framework helps with big data processing and analytics with its standard API. He also discusses how Spark compares with traditional MapReduce implementation like Apache Hadoop.
This article shows how to use Amazon DynamoDB to create a Mars Rover application. You can use the same concepts described in this post to build your own web application. 1
When it comes to database change, agility through automation - the ability to rapidly to accelerate delivery – is what differentiates world-class enterprises from the rest of the crowd. 4
GridGain announced that the In-Memory Data Fabric has been accepted into Apache Incubator program as Apache Ignite. InfoQ spoke with Nikita Ivanov about their product becoming part of Apache.
Application Lifecycle Management has traditionally been difficult for databases. Ben Rees, explains why the road ahead is now clear for Database Lifecycle Management.
The new “Hadoop in Practice. 2 Edition" book by Alex Holmes covers a lot of topics building Hadoop code and organizing data to support code simplicity and execution speed.
Datameer, a big data analytics application for Hadoop, introduced Datameer 5.0 with Smart Execution to enhance the data analytics. InfoQ spoke with Matt Schumpert from Datameer about the new product.
The article describes the general outline of the Stats Anomalies Detector developed at MyHeritage and provides a detailed explanation of how to enhance the code to meet your company’s needs.
The holy grail of database development is the ability to treat database objects as if they were normal source code. While SQL Server Data Tools doesn’t quite that level, it gets very close.
"Analytics Across the Enterprise" book is a collection of experiences by analytics practitioners in IBM. InfoQ spoke with authors about lessons learned and IBM technologies in the Big Data area.