Velocity: Microsoft's Distributed In-Memory Cache

Distributed in-memory caches have been rather popular over the last few years in everything from mainstream Java applications to the fringe languages like Erlang. Continuing its rather frantic efforts to catch up with technologies predominately found in the open source world, Microsoft has introduced its own distributed cache.

Velocity is a distributed cache designed specifically for the .NET platform. Many of the features will be recognizable to those familiar with other distributed in-memory caches. It is currently available as a community tech preview.

There are two types of clients for Velocity. Simple clients only know about a single cache server. If the requested object is not available on that cache server, the cache server will fetch it from the appropriate server. Routing clients are much more involved. They always know where a particular object lives so they can query that cache server directly. The performance impact of sending all of the cache location data to a routing client hasn't been discussed. In addition to the cache servers, both types of clients support a local cache option. This still requires checking the server for stale data, but should reduce network traffic when dealing with large cache objects.

For concurrency two options exist. With optimistic concurrency, the first update wins and any further updates to the now stale object will fail. With pessimistic locking, a lock handle is returned. Until unlocked, or the timeout expires, all attempts to gain a lock will fail. Failure to obtain a lock is a non-blocking operation.

Objects can be removed from the cache explicitly, via a expiration date, or whenever memory pressure is exceeded. This last method, known as eviction, uses a least recently used algorithm.

In addition to a key, objects may have a collection of tags associated with them. There are methods to retrieve one or all objects that match a list of tags.

While there is support for ASP.NET's Session model, that is only one of many uses Microsoft envisions. S Muralidhar writes,

Our support for ASP.NET is part of the integrated .NET platform story. Rest assured that we are not focusing exclusively on ASP.NET applications alone. As an example, we plan to integrate with plain .NET applications (windows service, for example) or  IIS applications that may not involve ASP.NET.

Now onto upcoming functionality. The current CTP features support for scale-out, local caching  and ASP.Net SessionState integration among many others. We have a full bag of work items we’re looking at for our subsequent CTPs and RTM – including support for availability, replicated caches, notifications, better management support etc.

Push-based notifications is a request we’ve heard from many folks. This is certainly an area we’re looking very deeply into. While our current CTP does not have this support, this is very likely to be remedied in our upcoming releases. In the interim, if you’re using Velocity with a local cache, we have some workarounds for this with APIs like GetIfVersionMismatch() to help deal with potentially stale/out-of-date objects in the local cache. (You will need to use a combination of Get() and GetIfVersionMismatch() to get the right behavior)

Support for more advanced techniques is also planned. Anil Nori adds,

> As applications start using the caches for data access, I also believe, they will demand richer data services like query, transactions, analytics, synchronization etc. For example, we believe .NET applications will require LINQ queries on the distributed cache, the same way they query the backend SQL Server database. We envision “Velocity” becoming such a comprehensive distributed caching platform.