InfoQ Homepage OLAP Content on InfoQ
News
RSS Feed-
Inside Uber’s Query Architecture: Simplifying Layers and Improving Observability
Uber rebuilt its Apache Pinot query architecture, replacing the Presto-based Neutrino system with a lightweight proxy called Cellar and Pinot’s Multi-Stage Engine Lite Mode. The redesign simplifies SQL execution, improves resource management, and ensures predictable performance for large-scale analytics workloads.
-
Google Spanner Unifies OLTP and OLAP with Columnar Engine
Google Spanner now features a columnar engine, allowing its distributed database to handle both OLTP and OLAP workloads on a single platform. This hybrid architecture eliminates the need for separate data warehouses and ETL pipelines. The engine's columnar storage and vectorized execution accelerate analytical queries up to 200x on live data, which is especially beneficial for AI applications.
-
Cloudflare Chooses PostgreSQL Extension over Specialized OLAP for 100K Row/Second Analytics
In a recent article from the engineering team behind the Zero Trust product suite, Cloudflare explains why it chose TimescaleDB over ClickHouse to add analytics and reporting capabilities to its internal platform. The author highlights the “phenomenal balance” between the simplicity of storing analytical data alongside configuration data and the performance of a specialized OLAP system.
-
HTAP: the Rise and Fall of Unified Database Systems?
A recent article by Zhou Sun sparked a debate in the data community about the future of HTAP systems. Hybrid transaction/analytical processing was meant to help integrate historical and online data at scale, supporting more flexible query methods and reducing business complexity.
-
Databases in 2024: Growth, Change and Controversy
Andrew Pavlo’s annual retrospective on the database world has recently been released, covering trends and innovations from the past year. The opinionated report, "Databases in 2024: A Year in Review," highlights that while we may indeed be in the "golden era of databases," last year brought significant license changes, the rapid growth of DuckDB, and some surprising new releases.
-
Apache Pinot 1.0 Provides a Realtime Distributed OLAP Datastore
Apache Pinot is an open source column-oriented distributed data store written in Java. Pinot is designed to use Online Analytical processing (OLAP) in order to answer multi-dimensional analytical (MDA) queries with low latency.
-
Instacart Creates a Self-Serve Apache Flink Platform on Kubernetes
Instacart moved their Apache Flink workloads from AWS EMR to Kubernetes to meet the high demand for data processing use cases using Flink within the organization, as using EMR became problematic for many teams with different requirements. As a result, they made the platform easier to use and reduced their operational and infrastructure costs.
-
Grab Shared Its Experience in Designing Distributed Data Platform
GrabApp is an application that customers select and buy their daily needs from merchants. To be scalable and manageable the data platform and ingestion should be designed as a distributed, fault-tolerant. To design this data platform two classes of data stores are considered: OLTP and OLAP.
-
Fitting Presto to Large-Scale Apache Kafka at Uber
The need for ad-hoc real-time data analysis has been growing at Uber. They run a large Apache Kafka deployment and need to analyse data going through the many workflows it supports. Solutions like stream processing and OLAP datastores were deemed unsuitable. An article was published recently detailing why Uber chose Presto for this purpose and what it had to do to make it performant at scale.
-
From Natural Language Queries to Insights: GCP BigQuery Data QnA Usage in Twitter
The Twitter engineering team has shared architectural details of their Qurious data insights platform and its advantages for real-time analysis. Designed for internal business customers, the platform allows users to analyze Twitter’s BigQuery data using natural language queries and create dashboards.
-
Improving Azure SQL Database Performance Using In-Memory Technologies
In late 2016, Microsoft announced the general availability of Azure SQL Database In-Memory technologies. In-Memory processing is only available in Azure Premium database tiers and provides performance improvements for On-line Analytical Processing (OLTP), Clustered Columnstore Indexes and Non-clustered Columnstore Indexes for Hybrid Transactional and Analytical Processing (HTAP) scenarios.
-
Olap4j 1.0: a Java API for OLAP Servers
Business Intelligence vendor Pentaho has announced the release of olap4j 1.0, a new, common Java API for any online analytical processing (OLAP) server.
-
Column-based Storage in SQL Server 2011
Imagine ad hock data mining queries against a single table with 1 TB of data and 1.44 billion rows coming back in roughly a second. This is the scenario Microsoft intends to support using 32-core machines and their new column-based storage engine.