InfoQ Homepage Distributed Data Content on InfoQ
-
Beyond Embedded: How DuckDB v2.0 Shifts Architecture Toward Distributed Network Capabilities
DuckDB Labs has previewed DuckDB v2.0, codenamed "Cyanoptera." This release includes over 10000 commits and introduces a client/server mode, enabling network connections. Improvements also encompass extension portability, advanced data types, and a new parser. Performance enhancements include asynchronous I/O and storage optimisations. General availability is expected in fall 2026.
-
Harper Argues against the Multi-System Stack and Releases 5.2
The database platform Harper advocates for a single-runtime architecture that keeps application code and data together, with its benchmark against a Vercel-based stack reporting significantly better performance on live, personalized-data workloads. Harper recently released version 5.2, with a new record cache and more throughput per node.
-
The Path to Sovereign Data: Challenges and Priorities in Local-First Computing
A panel on data ownership challenged the definition of "ownership," arguing it must extend beyond simple account control to include structural independence, interoperability, and community governance. Speakers like Zenna Fiscella, Paul Frazee, Boris Mann, and Robin Berjon emphasised the need for shared standards, unbundled platforms, and better tools to support user sovereignty.
-
How to Build More Resilient Local-First Applications with AT Protocol Infrastructure
Jake Lazaroff discussed the AT Protocol as a framework for distributed applications beyond social networking. He emphasised a local-first architecture where users maintain data in PDSs while leveraging shared infrastructure for synchronisation and updates. The presentation included experiments showcasing collaborative tools and highlighted the benefits of reduced reliance on app-specific backends.
-
DuckDB Quack: Client/Server Protocol over HTTP for Multi-User Analytics
DuckDB has recently announced Quack, a new remote protocol over HTTP that lets multiple DuckDB instances connect to and work with the same database over a network. The protocol introduces client-server capabilities to a database that was previously mostly local and embedded.
-
Dropbox Redesigns Compaction to Reclaim Space from Underfilled Storage Volumes
Dropbox recently explained how it improved storage efficiency in Magic Pocket, the company's internal immutable blob store for storing user files at scale, by redesigning compaction strategies to reclaim space from severely underfilled storage volumes. The system now periodically reorganizes valid data into new volumes, allowing old, partially used ones to be cleared and reused.
-
Etsy Migrates 1000-Shard, 425 TB MySQL Sharding Architecture to Vitess
The Etsy engineering team recently described how the company migrated its long-running MySQL sharding infrastructure to Vitess. The transition moved shard routing from Etsy’s internal systems to Vitess using vindexes, enabling capabilities such as resharding data and sharding previously unsharded tables.
-
Google BigQuery Previews Cross-Region SQL Queries for Distributed Data
Google Cloud has recently announced the preview of a global queries feature for BigQuery. The new option lets developers run SQL queries across data stored in different geographic regions without first moving or copying the data to aggregate the results.
-
Uber Moves from Static Limits to Priority-Aware Load Control for Distributed Storage
Uber engineers detailed how they evolved their storage platform from static rate limiting to a priority-aware load management system. The approach protects Docstore and Schemaless, Uber’s MySQL-based distributed databases, by colocating control with storage, prioritizing critical traffic, and dynamically shedding load under overload conditions.
-
Amazon Aurora DSQL Goes GA: Distributed, PostgreSQL-Compatible Serverless Database
Amazon has recently announced the general availability of Amazon Aurora DSQL, a PostgreSQL-compatible, serverless, and distributed database. The new managed service is designed to support active-active high availability and multi-region strong consistency.
-
Google Cloud Introduces HDD Tier for Spanner Database, Cutting Cold Storage Costs by 80%
Google has recently introduced tiered storage for Spanner, its distributed SQL database on Google Cloud. This tiered storage is based on a new HDD storage option that is 80% cheaper than the existing SSD option, allowing for cost optimization of older data while minimizing the overhead associated with traditional data migration.
-
Cloud-Native Distributed Storage System CubeFS Graduates from CNCF
The Cloud Native Computing Foundation (CNCF) has announced that the open-source distributed storage system CubeFS has reached graduation status. CubeFS is a storage solution supporting multiple access protocols, including POSIX, HDFS, S3, and its own REST API. CubeFS key platform areas are big data, AI/LLM applications, container platforms, and databases.
-
Uber's CacheFront: Powering 40M Reads per Second with Significantly Reduced Latency
Uber developed an innovative caching solution, CacheFront, for its in-house distributed database, Docstore. CacheFront enables over 40M reads per second from online storage and achieves substantial performance improvements, including a 75% reduction in P75 latency and over 67% reduction in P99.9 latency, demonstrating its effectiveness in enhancing system efficiency and scalability.
-
Apache Pinot 1.0 Provides a Realtime Distributed OLAP Datastore
Apache Pinot is an open source column-oriented distributed data store written in Java. Pinot is designed to use Online Analytical processing (OLAP) in order to answer multi-dimensional analytical (MDA) queries with low latency.
-
Distributed Materialized Views: How Airbnb’s Riverbed Processes 2.4 Billion Daily Events
Airbnb created Riverbed, a Lambda-like data framework for producing and managing distributed materialized views. The framework supports over 50 read-heavy use cases where data is sourced from multiple data sources within the company’s service-oriented architecture (SOA) platform. It uses Apache Kafka and Apache Spark for online and offline components, respectively.