Big Data Follow 147 Followers

Data Workflow Management Using Airbnb's Airflow

by Alex Giamas Follow 10 Followers on  Sep 08, 2015

Airbnb recently opensourced Airflow, its own data workflow management framework. Airflow is being used internally at Airbnb to build, monitor and adjust data pipelines. Airflow’s creator, Maxime Beauchemin and Agari’s Data Architect and one of the framework’s early adopters Siddharth Anand discuss about Airflow, where it can be of use and future plans.


Software Defined Data Mart In The Enterprise Using Metanautix Quest

by Alex Giamas Follow 10 Followers on  Jun 29, 2015

Metanautix recently announced the newest version of its product, Quest. Quest allows enterprises to build software defined data marts that can run in virtualized servers. Designed from the ground up with security and auditability in mind, Quest can deal with Big Data workloads and encapsulate it into different logical views, ready for consumption by different departments in enterprise.


Thoughtworks Technology Radar March 2012

by Craig Smith Follow 6 Followers on  Mar 22, 2012 7

ThoughtWorks recently published the latest update to its Technology Radar; a report produced to help technology decision makers understand emerging trends in software development techniques, tools, languages and platforms. There are some interesting observations of interest to Agile software development teams.


What’s New in SQL Server 2012 RC0

by Abel Avram Follow 11 Followers on  Nov 18, 2011

Microsoft has released SQL Server 2012 Release Candidate 0. There are many new features, including: AlwaysOn, better performance management, more reporting and visualization tools, Columnstore index, and FileTables. The product will come in 3 main editions: Standard, Business Intelligence and Enterprise.


Olap4j 1.0: a Java API for OLAP Servers

by Charles Humble Follow 929 Followers , Jai Hirsch Follow 0 Followers on  Jun 24, 2011

Business Intelligence vendor Pentaho has announced the release of olap4j 1.0, a new, common Java API for any online analytical processing (OLAP) server.


Column-based Storage in SQL Server 2011

by Jonathan Allen Follow 608 Followers on  Mar 07, 2011 4

Imagine ad hock data mining queries against a single table with 1 TB of data and 1.44 billion rows coming back in roughly a second. This is the scenario Microsoft intends to support using 32-core machines and their new column-based storage engine.


Better Developer Experience in Version 1.5 of the Data Access Framework MetaModel

by Michael Stal Follow 0 Followers on  Feb 22, 2011 1's open-source Java framework MetaModel implements a unified API for the access, exploration, and query of different datastores., both a website and an open source software organization dedicated to "the development of Open Source software related to Business Intelligence and Data Warehousing", has recently published version 1.5 of MetaModel.


A Case for Graph Databases

by Jonathan Allen Follow 608 Followers on  Sep 15, 2010 3

We talk with Daniel Kirstenpfad, founder and CTO of sones GmbH, about Graph Databases and how they can better model some types of data such as relations in a social networking application. A graph database can offer performance benefits over other types of databases because they explicitly represent a graph and are organized to have index free adjacency.


LinkedIn's Data Infrastructure

by Ron Bodkin Follow 0 Followers on  Aug 04, 2010 1

Jay Kreps of LinkedIn presented some informative details of how they process data at the recent Hadoop Summit. Kreps described how LinkedIn crunches 120 billion relationships per day and blends large scale data computation with high volume, low latency site serving.


Facebook on Hadoop, Hive, HBase, and A/B Testing

by Ron Bodkin Follow 0 Followers on  Jul 14, 2010 1

The Hadoop Summit of 2010 included presentations from a number of large scale users of Hadoop and related technologies. Notably, Facebook presented a keynote and details information about their use of Hive for analytics. Mike Schroepfer, Facebook's VP of Engineering delivered a keynote describing the scale of their data processing with Hadoop.


Emergent Data Architectures Highlights From GigaOm Structure Conference

by Ron Bodkin Follow 0 Followers on  Jul 09, 2010

The GigaOM Stucture conference a couple of weeks ago addressed many areas of cloud computing. One of the key themes of the event was the emergence of new data architectures. Throughout the panels, interviews, and presentations many speakers identified significant changes in how data gets handled that will be coming.


Mahout 0.3: Open Source Machine Learning

by Gilad Manor Follow 0 Followers on  Apr 19, 2010 1

The need for machine-learning techniques like clustering, collaborative filtering, and categorization has steadily increased the last decade along with the number of solutions needing quick and efficient algorithms to transform vast amounts of raw data into relevant information. Apache Mount 0.3 has been announced on March, adding more functionality, stability and performance.


Information Can Be Sold and Bought in “Dallas”

by Abel Avram Follow 11 Followers on  Dec 24, 2009 1

Microsoft’s service codename “Dallas” is an information marketplace bringing together data, imagery and service providers and their consumers facilitating information exchange through a single point of access.


MapPoint Add-In For SQL Server Released

by Al Tenhundfeld Follow 0 Followers on  Aug 22, 2009

Microsoft released a free MapPoint 2009 Add-In for SQL Server 2008 spatial data. The add-in can be used with MapPoint to build map graphics against queries on SQL Server 2008 spatial geography columns.


Is the NoSQL Meeting Announcing the End of the RDBMS Era?

by Abel Avram Follow 11 Followers on  Aug 19, 2009 13

The NoSQL meeting tried to raise the awareness towards the opportunity of using non-relational databases which promise to be cheaper, simpler to administer and maintain, and offering superior scalability. Michael Stonebraker, co-creator of Ingres and Postgres, thinks that the end of RDBMS era is close, while others think that we are not there yet.

Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you