InfoQ Homepage Big Data Content on InfoQ
-
Google on the Technical Debt of Machine Learning
A number of Google researchers and engineers presented their view on the technical debt of using machine learning at a NIPS workshop. They identified different aspects of technical debt and came to the conclusion that without proper care, using machine learning or complex data analysis in your company can induce new kinds of technical debt different from classical software engineering.
-
Apache Spark 1.2.0 Supports Netty-based Implementation, High Availability and Machine Learning APIs
Apache Spark 1.2.0 was released with Netty-based implementation, High Availability and Machine Learning APIs. It represents the work of 172 contributors from over 60 institutions and comprises more than 1000 patches. InfoQ talks with Patrick Wendell, a Spark committer and PMC member.
-
Splunk Enterprise 6.2 Supports Instant Pivot and Enhanced Event Pattern Detection
The latest version of big data analytics tools Splunk Enterprise and Hunk support instant pivot, enhanced event pattern detection, and prebuilt dashboard panels. Splunk Inc., provider of the software platform for operational intelligence, recently announced the general availability (GA) of version 6.2 of Splunk Enterprise and Hunk: Splunk Analytics for Hadoop and NoSQL Data Stores.
-
New and Interesting on ThoughtWorks Radar Jan 2015
ThoughtWorks has published a digital preview of the January 2015 radar, providing opinion on techniques, tools, platforms and languages and taking a snapshot of the current trends in software technology.
-
Splice Machine Version 1.0 Supports Integration with Hadoop and Analytic Window Functions
Splice Machine version 1.0 supports analytic window functions and integration with Hadoop ecosystem. Splice Machine team recently released their Hadoop based RDBMS data management solution that can be used for transactional workloads on Hadoop.
-
Google Open Sources Cloud Dataflow Java SDK
Google announced earlier this year their Cloud Dataflow, a service and SDK for processing large amounts of data in batches or real time. Now they have open sourced the Dataflow Java SDK, enabling developers to see how it works and possibly use the SDK for services running on-premises or in other clouds.
-
LinkedIn Open Sources Cubert With an Eye To Big Data Analytics
LinkedIn recently open sourced Cubert, its High Performance Computation Engine for Complex Big Data Analytics. Cubert is a framework written for analysts and data scientists in mind.Developed completely in Java and expressed as a scripting language, Cubert is designed for complex joins and aggregations that frequently arise in the reporting world.
-
Agile View of Big Data
An agile view of Big Data, wherein data is viewed as a real time stream, offers a new look at how data is managed. Using an agile data infrastructure, organizations can conquer Big Data challenges with a level of ease, flexibility and performance. White paper by codeFutures describes the Agile view of Big Data.
-
Gobblin, LinkedIn's Unified Data Ingestion Platform
At the 2014 QCon San Francisco conference, LinkedIn's Lin Qiao gave a talk on their Gobblin project (also summarized in a blog post) that is a unified data ingestion system for their internal and external data sources.
-
GridGain Becomes Apache Ignite
GridGain's In-Memory Data Fabric entered Apache Incubator last October under the name of Apache Ignite. The company donated its flagship in-memory computing platform to the Apache Software Foundation with the intention of attracting external developers and growing a viable community around its core technology.
-
IBM, Databricks, GraphLab Present Notebooks as Unified Interfaces for Building Prediction Apps
At the StrataHadoop conference in Barcelona last week, Rod Smith, Vice President of the IBM Emerging Internet Technologies organization, presented work on an internal product they have been developing in their consulting work with clients that integrates data sources, and data analysis.
-
Mahout to Get Self-Optimizing Matrix Algebra Interface with Pluggable Backends for Spark and Flink
At the recent GOTO conference in Berlin, Mahout committer Sebastian Schelter outlined recent advances in Mahout's ongoing effort to create a scalable foundation for data analysis that is as easy to use as R or Python.
-
Web Summit 2014 Day Two Review
Yesterday concluded the second day of the Web Summit in Dublin, Ireland. We see what happened and what is new from last day at the event.
-
Web Summit 2014 Day One Review
Web Summit, one of the largest technology conferences in Europe opened up today. Famous people from the technology and business world are expected to talk, like Peter Thiel, Drew Houston and Anna Patterson.
-
Forrester Wave: Evaluating NoSQL Key-Value Databases
In their first Forrester Wave: NoSQL Key-Value Databases, released in Q3 2014, Forrester has evaluated the most popular NoSQL database offerings.