BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Olson and Söderqvist Discuss Valkey’s Evolution and Future Past Caching Use Cases at OSS EU

Olson and Söderqvist Discuss Valkey’s Evolution and Future Past Caching Use Cases at OSS EU

Listen to this article -  0:00

More than two years after the fork from Redis, Valkey looks like a young but growing project that aims to gain more momentum. With twelve maintainers and nine technical steering committee members spread across eight companies (among which are Ericsson, AWS, Apple and Percona), and multiple community conferences reaching up to one hundred participants. During Open Source Summit Europe, InfoQ sat with two of its initial developers to find out more about the evolution and future of the project.

InfoQ: Thank you for taking the time to answer some questions for our readers. Let's start by introducing yourselves.

Olson: I am Madelyn Olson, and I helped create the fork after the license change. I work at AWS and serve on Valkey’s technical steering committee (TSC). Before Valkey, I was a Redis committer as well.

Söderqvist: I am Viktor Söderqvist, and I work on open source at Ericsson and was also involved in creating the fork.

InfoQ: What did the 8.x releases change technically?

Olson: The theme was foundational infrastructure. Async I/O threading increased throughput from approximately 200,000-250,000 requests per second to around one million per process. Unlike the previous I/O-thread arrangement, thread configuration can be changed at runtime, allowing operators to add cores to help handle burst workloads. Versions 8.0 and 8.1 also included memory-efficiency work involving key embedding and the per-slot dictionary.

Dual-channel replication addressed pressure on a primary during replica synchronisation. Previously, a full copy of the dataset and the subsequent stream of replication changes were sent sequentially. Sending them in parallel reduces the memory pressure associated with that process.

InfoQ: Why were the version 9 cluster changes important?

Olson: Cluster mode distributes keys by slot across nodes, but moving to it had two significant limitations. Cross-slot operations require data to reside on the same shard, and cluster mode previously did not support numbered databases, the namespaces available in single-instance deployments. Adding numbered databases removed one reason some users could not migrate to cluster mode.

Atomic slot migration addressed another problem: moving data key by key could leave a cluster partially migrated if the operation failed. The new approach pre-stages the data and then switches over, so the migration either takes effect or does not. Version 9 also introduced hash-field expiration, described as one of the project’s most requested features. The interviewees cited Mastodon-style deployments, multiple servers using one cluster, as an example of demand for the cluster changes, while cautioning against treating that arrangement as strong multi-tenancy.

InfoQ: What problem is the GLIDE client library designed to solve?

Olson and Söderqvist: Redis clients originally had a relatively simple task: communicate with a single node. Cluster topology, TLS, redirects, reconnects, and connection pooling made client behaviour more complex, and different language clients implemented those features inconsistently. Glide, which began as an AWS project, puts connection and topology handling in a shared Rust core and exposes it through language-specific bindings. A bug fixed in the core can therefore be fixed for all the bindings that use it.

WebAssembly was considered, but the team judged it too early for the approach at the time. Glide is not intended to eliminate native alternatives: Valkey continues to support other clients, including Go implementations, for users whose needs are already met by them.

InfoQ: What could take Valkey beyond its current caching use cases?

Olson and Söderqvist: One proposed direction is synchronous replication, with the aim of providing durability guarantees for data that is not merely ephemeral. Another is tiering: keeping hot data in memory while moving colder data to SSD or other storage.

Olson: Tiering may need more than one strategy. Small, infrequently accessed keys could be moved out of DRAM; sufficiently large objects might instead remain on disk and be streamed directly to the network. The team is considering a more general interface that could support backends beyond local NVMe, such as S3 or Postgres. Parquet was discussed as potentially useful for change-data-capture or clickstream output, rather than as the format for dynamically reading and rewriting tiered data. Cross-cluster active-active replication with eventual consistency remains a longer-term idea, not an implementation commitment.

Besides features, the project is also investing in release and security work. Automated backporting is intended to shorten the time needed to prepare fixes across releases, while AI-assisted pre-review and proactive security testing are being used to identify problems before a release.

About the Author

Rate this Article

Adoption
Style

BT