InfoQ Homepage Key-Value Store Content on InfoQ
-
Airbnb Adds Adaptive Traffic Control to Manage Key Value Store Spikes
Airbnb upgraded Mussel, its multi-tenant key-value store, replacing static per-client rate limits with an adaptive, resource-aware traffic control system. The redesign ensures resilience during traffic spikes, protects critical workflows, and maintains fair usage across thousands of tenants while scaling efficiently.
-
Airbnb’s Mussel V2: Next-Gen Key Value Storage to Unify Streaming and Bulk Ingestion
Airbnb’s engineering team re-architected its internal key-value storage system, Mussel, to unify streaming and bulk ingestion while simplifying operations, achieving over 100,000 writes per second and sub-25ms read latencies on 100-terabyte tables, while leveraging Kubernetes, Kafka, and a NewSQL backend to improve scalability, reliability, and operational efficiency across its internal services.
-
Cloudflare Rearchitects Workers KV Following GCP Outage, Achieves 40x Performance Gain
Cloudflare has recently redesigned Workers KV with a hybrid storage architecture that automatically routes objects between distributed databases and object storage based on size characteristics, while operating dual storage backends. This change improved the p99 read latencies from 200ms to under 5ms for their global key-value store while handling hundreds of billions of key-value pairs.
-
How Cloudflare Migrated Quicksilver to Multi-Level Caching While Serving Billions of Requests
The engineering team at Cloudflare recently shared how they transitioned Quicksilver, their internal global key-value store, to a tiered caching architecture. They described their incremental journey from storing everything everywhere to adopting a distributed caching system, improving storage efficiency while preserving consistency guarantees and low-latency reads at the edge.
-
Redis Returns to Open Source under AGPL License: Is It Too Late?
Redis 8 has recently hit general availability, switching to the AGPLv3 license. A year after leaving its open source roots to challenge cloud service providers and following the birth of Valkey, Redis has rehired its creator and moved back to an open source license.
-
Valkey 8.0 Now Generally Available with Improved Memory Efficiency
The Linux Foundation has announced the general availability of Valkey 8.0, the open source in-memory storage solution developed as a successor to Redis. By introducing a dictionary per slot and embedding keys directly into dictionary entries, developers can achieve up to 20% more capacity, allowing for the storage of additional keys per node.
-
Netflix’s Pushy: Evolution of Scalable WebSocket Platform That Handles 100Ms Concurrent Connections
Netflix shared details on the evolution of Pushy, a WebSocket messaging platform that supports push notifications and inter-device communication across many different devices for the company’s products. Netflix’s engineers implemented many improvements across the Pushy ecosystem to ensure the platform's scalability and reliability and support new capabilities.
-
Async Ops and Scalable Keyspaces Make Threads Go Viral
Meta's Engineering team has published a post explaining how they built the infrastructure for Threads, their new online social media and networking service. The decision to launch was abrupt, with the infrastructure teams having just two days' notice. However, the teams were confident in Meta's infrastructure's maturity and past performance to effectively support the app's rapid growth.
-
AWS Introduces Amazon CloudFront KeyValueStore: a Low Latency Datastore for CloudFront Functions
AWS recently announced Amazon CloudFront KeyValueStore, a secure global low-latency key-value datastore that allows read access from within CloudFront Functions, enabling advanced customizable logic at the CloudFront edge locations.
-
JunoDB: PayPal Open Sources Key-Value Store Powering 350 Billion Daily Requests
PayPal recently open-sourced JunoDB, a distributed key-value store that uses RocksDB as the underlying storage engine. A highly available and security-focused database, JunoDB serves daily 350 billion requests at PayPal.
-
Amazon DynamoDB - Evolution of a Hyper-Scale Cloud Database Service: Akshat Vig at QCon SF 2022
Akshat Vig, a principal engineer in NoSQL databases at Amazon, spoke at QCon San Francisco about Amazon DynamoDB: Evolution of a Hyper-Scale Cloud Database Service. The talk was part of the "Architectures You've Always Wondered About" track.
-
How Pinterest Scaled up Its Ad-Serving Architecture
On the Pinterest Engineering Blog, Nishant Roy wrote about their strategy to overcome a scaling problem with their ad corpus. Their existing solution had hit its scaling limit, but further growth was necessary. Its redesign offloaded the ad index to a key-value store and optimised garbage collection in their Go applications to increase their ad corpus size by a factor of 60.
-
ZippyDB: the Architecture of Facebook’s Strongly Consistent Key-Value Store
Facebook Engineering recently published how it built its general-purpose key-value store, known as ZippyDB. ZippyDB is Facebook's biggest key-value store, which has been in production for more than six years. It offers flexibility to applications in terms of tunable durability, consistency, availability, and latency guarantees.
-
Distributed Key-Value Store etcd Graduates at CNCF
The CNCF announced the graduation of the etcd project - a distributed key-value store used by many open source projects and companies. Originally written at CoreOS, etcd was accepted into the CNCF incubation stage in 2018 and graduated in November 2020.
-
Cloudflare Releases Workers KV, a Serverless Key-Value Store at the Edge
What sits between your application and the client's browser? When the answer is "the internet", Cloudflare wants their Workers platform to play a part. They recently expanded that platform with Workers KV, a distributed, eventually-consistent key-value store available in 180+ edge locations.