InfoQ Homepage Data Storage Content on InfoQ
-
DuckLake 1.0: Data Lake Format with SQL Catalog Metadata
DuckDB Labs recently released DuckLake 1.0, a data lake format that stores table metadata in a SQL database rather than across many files in object storage. The first implementation is available as a DuckDB extension and includes catalog-stored small updates, improved sorting and partitioning options, and compatibility with Iceberg-style data features.
-
Dropbox Redesigns Compaction to Reclaim Space from Underfilled Storage Volumes
Dropbox recently explained how it improved storage efficiency in Magic Pocket, the company's internal immutable blob store for storing user files at scale, by redesigning compaction strategies to reclaim space from severely underfilled storage volumes. The system now periodically reorganizes valid data into new volumes, allowing old, partially used ones to be cleared and reused.
-
AWS Introduces S3 Files, Bringing File System Access to S3 Buckets
AWS recently introduced S3 Files, which lets users mount an Amazon S3 bucket and access its data through a standard file system interface. Applications can read and write files using standard file operations, while the system automatically translates them into S3 requests, allowing compute services to work directly with data stored in S3.
-
Google Cloud Introduces Bigtable Tiered Storage
Google Cloud recently introduced the preview of Bigtable tiered storage. The new feature allows developers to manage both hot and cold data within a single Bigtable instance, optimizing costs while maintaining access to all data.
-
Amazon Timestream for InfluxDB Adds Support for InfluxDB 3 Core and Enterprise
InfluxData has launched InfluxDB 3 Core and Enterprise on Amazon Timestream, offering a high-speed, open-source time-series database for real-time applications. With enhanced security, scalability, and performance, developers can seamlessly integrate with AWS services. InfluxDB 3 redefines data management for AI-driven environments, enabling rapid analytics and decision-making.
-
Meta Open Sources OpenZL: a Universal Compression Framework for Structured Data
Meta’s OpenZL changes the way data is compressed by maximizing efficiency for structured datasets, outperforming traditional methods like Zstandard. With a universal decompressor and custom compression plans, it simplifies operational deployment while achieving superior compression ratios and speeds, making it an essential tool for modern data infrastructures.
-
Datadog Launches Monocle, a Unified Rust-Powered Real-Time Metrics Engine
Datadog has launched Monocle, a new real-time time series storage engine written in Rust. The system unifies the company’s metrics storage infrastructure, delivering higher ingestion throughput and lower query latency while reducing operational complexity. Monocle replaces several generations of storage backends, addressing concurrency challenges and scaling limits that accumulated over time.
-
Google Cloud Introduces Non-Disruptive Cloud Storage Bucket Relocation
Google Cloud's innovative Cloud Storage bucket relocation feature enables seamless, non-disruptive data migration across regions while preserving metadata and minimizing application downtime. Maintain governance, enhance lifecycle management, and leverage insights for optimized storage—all without altering access paths. Experience efficient, low-latency solutions tailored for your needs.
-
Google Cloud Announces Rapid Storage for Millisecond-Latency Workloads
At the recent Google Cloud Next 2025, the cloud provider announced Rapid Storage, a new Cloud Storage zonal bucket designed to deliver consistent single-digit millisecond data access for frequently accessed data and latency-sensitive applications. The new storage class provides under 1ms random read and write latency, 20x faster data access, and 6 TB/s of throughput.
-
Improving Distributed System Data Integrity with Amazon S3 Conditional Writes
AWS recently announced support for conditional writing in Amazon S3, allowing users to check for the existence of an object before creating it. This feature helps prevent overwriting existing objects when uploading data, making it easier for applications to manage data.
-
Microsoft Expands Azure Data Box Capabilities for Enhanced Offline Data Migration
Microsoft recently announced several capabilities for its Azure Data Box, a service that has been available since 2019 and facilitates offline data migration to Azure. These new capabilities enhance data transfer speed, flexibility, and security, offering organizations more efficient ways to move large datasets to the cloud without relying solely on network bandwidth.
-
Redis Improves Performance of Vector Semantic Search with Multi-Threaded Query Engine
Redis, the in-memory data structure store, has recently released its enhanced Redis Query Engine. This comes at a time when vector databases are gaining prominence due to their importance in retrieval-augmented generation (RAG) for GenAI applications. Redis announced significant improvements to its Query Engine, using multi-threading to enhance query throughput while maintaining low latency.
-
Stateful Cloud Services at Neon Navigating Design Decisions and Trade-Offs: Q&A with John Spray
At QCon London, John Spray, a storage engineering lead @neon.tech, discussed the often-overlooked complexities of stateful cloud service design, using Neon Serverless Postgres as a case study. His session was part of the Cloud-Native Engineering track on the first day of the conference, and InfoQ carried out an interview.
-
Amazon RDS Introduces Faster Storage for High-Performance Database Workloads
AWS has recently introduced support for io2 Block Express volumes on Amazon RDS. Priced as the existing Provisioned IOPS (PIOPS) io1, the new io2 Block Express volumes are compatible with all database engines and are designed for high-performance, high-throughput, and low-latency database workloads.
-
Cost-Effective Solution for Infrequent Data Access and Retention with Azure Blob Storage Cold Tier
Microsoft recently announced the general availability of the Azure Blob Storage Cold Tier, an online tier designed explicitly for efficiently storing infrequently accessed or modified data while ensuring immediate availability.