Hekaton: In-Memory Transaction Processing Integrated with SQL Server

During the Professional Association of SQL Server (PASS) Summit 2012, a technical conference for SQL Server professionals, Microsoft announced Hekaton, an in-memory row-based data management system targeted at transaction processing (TP) workloads. Besides the advertised increase in TP speeds of up to 10x for old applications and up to 50x for new optimized ones, Microsoft touts Hekaton as being fully integrated into SQL Server.

Existing applications may benefit from the new database technology by determining the most used tables with the help of a tool, then setting the server to store those tables in a system’s main memory in memory-optimized data structures for faster access time, while the rest of data will reside in traditional storage-optimized structures that can be paged in or out to disk when necessary.

Until now, SQL Server has compiled queries and stored procedures into data structures that are interpreted by the query processor, but Hekaton compiles T-SQL stored procedures directly into native code for faster execution.

Hekaton uses a new concurrency control mechanism (PDF) developed by Microsoft and researchers from the University of Wisconsin using lock-free data structures for better scalability across multiple cores, avoiding locks while preserving ACID transaction integrity.

Hekaton is currently in tests by selected partners of Microsoft, and will be included in the next major version of SQL Server. When it is released, Hekaton will compete with SAP Hana, a stand alone appliance and database for real-time analytics released in 2010, and Oracle Exadata X3, also an appliance running Oracle Database 11g, which keeps hot data in solid-state storage.

Microsoft has announced a new version of SQL Server 2012 Parallel Data Warehouse (PDW), an SQL Server appliance, to be made available during first half of 2013.