BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Extreme Transaction Processing Patterns: Write-behind Caching

Posted by Lan Vuong on Dec 07, 2009 |

Introduction

Applications typically use a data cache to increase performance, especially where the application predominantly uses read-only transactions. These applications directly update the database for changes in the data. The issue here is that as the load increases then the response time on these updates grows. Databases are not good at executing lots of concurrent transactions with a small number of records per transaction. Databases are much better at executing batched transactions.

Eventually, the database will saturate the CPU or disks and at that point the response time will rise as additional load is added. Conventional in-memory caches are also limited to only storing what can fit in the free memory of a JVM. Once we need to cache more than this amount of data then thrashing occurs where the cache continuously evicts data to make room for other data. The required record must then be read continually thereby making the cache useless and exposing the database to the full read load.

There are several products currently available, including IBM® WebSphere® eXtreme Scale, Oracle Coherence, and Gigaspaces that allow all of the free memory of a cluster of JVMs to be used as a cache rather than just the free memory of a single JVM. This allows the capacity of the cache to scale as more JVMs are incorporated. If these JVMs are on additional physical servers with CPU, memory and network, then this allows scalable servicing of read requests. They can also provide scalable servicing of update requests by leveraging their write-behind technology. The scalability of write-behind caching makes it ideal to handle extreme transaction processing (XTP) scenarios. XTP is defined by Gartner as "an application style aimed at supporting the design, development, deployment, management and maintenance of distributed TP applications characterized by exceptionally demanding performance, scalability, availability, security, manageability and dependability requirements.”(1)

In this paper, we will illustrate how to optimize the performance of an application by leveraging the write-behind pattern with examples using IBM WebSphere eXtreme Scale. The write-behind function batches updates to the back-end database asynchronously within a user configurable interval of time. The obvious advantage of this scenario is reduced database calls and therefore reduced transaction load and faster access to objects in the grid. This scenario also has faster response times than the write-through caching scenario where an update to the cache results in an immediate update to the database. In the write-behind case, transactions no longer have to wait for the database write operation to finish. Additionally, it protects the application from database failure as the write-behind buffer will hold changes through memory replication until it can propagate them to the database.

Defining and Configuring Key Concepts

What is a “write-behind” cache?

In a write-behind cache, data reads and updates are all serviced by the cache, but unlike a write-through cache, updates are not immediately propagated to the data store. Instead, updates occur in the cache, the cache tracks the list of dirty records and periodically flushes the current set of dirty records to the data store. As an additional performance improvement, the cache does conflation on these dirty records. Conflation means if the same record is updated or dirtied multiple times within the buffering period then it only keeps the last update. This can significantly improve performance in scenarios where values change very frequently such as stock prices in financial markets. If a stock price changes 100 times a second then normally that would have meant 30 x 100 updates to the loader every 30 seconds. Conflation reduces that to one update.

With WebSphere eXtreme Scale, the list of dirty records is replicated to ensure its survival if a JVM exits and the customer can specify the level of replication by setting the number of synchronous and asynchronous replicas. Synchronous replication means no data loss when a JVM exits but it is slower as the primary must wait for the replicas to acknowledge they have received the change. Asynchronous replication is much faster but changes from the very latest transactions may be lost if a JVM exits before they are replicated. The dirty record list is written to the data source using a large batch transaction. If the data source is not available, then the grid continues processing requests and will try again later. The grid can offer reduced response times as it scales for changes because changes are committed to the grid alone and transactions can commit even if the database is down

A write-behind cache may not be suitable for all situations. The nature of write-behind means that, for a time, changes which the user sees as committed are not reflected in the database. This time delay is called “cache write latency” or “database staleness”; the delay between database changes and the cache being updated (or invalidated) to reflect them is called “cache read latency” or “cache staleness”. If all parts of the system access the data through the cache (e.g., through a common interface) then write-behind will be acceptable as the cache will always have the correct latest record. It is expected that for a system using write behind that all changes are made through the cache and through no other path.

Either a sparse cache or complete cache could be used with the write-behind feature. The sparse cache only stores a subset of the data and can be populated lazily. Sparse caches are normally accessed using keys since not all of the data is available in the cache and thus queries cannot be done using the cache. A complete cache contains all the data but could take a long time to load initially. A third method is available that is a compromise between these two options. It preloads the cache with a subset of the data in a short amount of time and then lazily loads the rest. The subset that is preloaded is roughly 20% of the total number of records but it fulfills 80% of the requests.

An application using a cache in this manner is typically only used for scenarios which access partitionable data models using simple CRUD (Create, Read, Update, and Delete) patterns.

Configuring the write-behind function

The write-behind function is enabled (in the case of WebSphere eXtreme Scale) in the objectgrid.xml configuration by adding the writeBehind attribute to the backingMap element as shown below. The value of the parameter uses the syntax “"[T(time)][;][C(count)]" which specifies when the database updates occur. The updates are written to the persistent store when either the specified time in seconds has passed or the number of changes in the queue map has reached the count value.

Listing 1. An example of write-behind configuration

<objectGrid name="UserGrid">

<backingMap name="Map" pluginCollectionRef="User" lockStrategy="PESSIMISTIC" writeBehind="T180;C1000"/> 

JPA loader

WebSphere eXtreme Scale uses loaders to read data from and write data to the database from the in-memory cache. Starting with WebSphere eXtreme Scale 6.1.0.3, there are two built-in loaders that interact with JPA providers to map relational data to the ObjectGrid maps, the JPALoader and JPAEntityLoader. The JPALoader is used for caches that store POJO’s and the JPAEntityLoader is used for caches that store ObjectGrid entities.

To configure a JPA loader, changes must be made to the objectgrid.xml and a persistence.xml file must be added to the META-INF directory. A loader bean is added to the objectgrid.xml along with the necessary entity class names. A transaction callback must be defined in the objectgrid.xml to receive transaction commit or rollback events and send it to the JPA layer. The persistence.xml file denotes a particular JPA provider for the persistence unit along with provider specific properties.

Exploring an Example Business Case

An online banking website with a growing number of users is experiencing slow response times and scalability issues with their environment. They need a way to support their clients with the existing hardware. Next we’ll walk you through this use case to see how the write-behind feature can help resolve their issue.

Use Case: Portal Personalization

Instead of pulling the user profile information directly from the database, they will preload the cache with the profiles from the database. This means the cache can service the read requests instead of the database. Profile updates were also written directly to the database in the old system. This limited the number of concurrent updates/second with an acceptable response time as the database machine would saturate. The new system writes profile changes to the grid and then these changes are pushed to the database using the write-behind technology. This allows the grid to service these with the usual grid quality of service and performance and it completely decouples the single instance database from the read and write profile operations. The customer can now scale up the profile service simply by adding more JVMs/servers to the grid. The database is no longer a bottleneck as there are vastly fewer transactions sent to the back-end. The quicker response leads to faster page loads and results in a better user experience as well as cost effective scaling of the profile server and better availability as the database is no longer a single point of failure.

In this case, we are using WebSphere eXtreme Scale with a DB2Ò database and the OpenJPA provider. The data model for this scenario is a User which contains one-to-many UserAccount’s and UserTransaction’s. The following code snippet from the User class shows this relationship:

Listing 2: Use case entity relationships

@OneToMany(mappedBy = "user", fetch = FetchType.EAGER, cascade = { Cas-cadeType.ALL })

private Set<UserAccount> accounts = new HashSet<UserAccount>();


    
@OneToMany(mappedBy = "user", fetch = FetchType.EAGER, cascade = { Cas-cadeType.ALL })

private Set<UserTransaction> transactions = new HashSet<UserTransaction>();

1. Populating the database

The sample code includes the class PopulateDB which loads some user data into the database. The DB2 database connection information is defined in the persistence.xml shown earlier. The persistence unit name listed in the persistence.xml is used to create the JPA EntityManagerFactory. User objects are created and then persisted to the database in batches.

2. Warming the cache

After the database is loaded, the cache is preloaded using data grid agents. The records are written to the cache in batches so there are fewer trips between the client and server. Multiple clients should also be used to speed up the warm-up time. The cache can be warmed up with a “hot” set of data which is a subset of the all the records and the remaining data will be loaded lazily. Preloading the cache increases the chances of a cache hit and reduces the need to retrieve data from back-end tiers. For this example, data matching the database records was inserted into the cache rather than loaded from the database to expedite the execution time.

3. Generating load on the grid

The sample code includes a client driver that mimics operations on the grid to demonstrate how the write-behind caching function increases performance. The client has several options to tweak the load behavior. The following command will load 500K records to the “UserGrid” grid using 10 threads with a rate of 200 requests per thread.

$JAVA_HOME/bin/java -Xms1024M -Xmx1024M -verbose:gc -classpath
$TEST_CLASSPATH ogdriver.ClientDriver -m Map -n 500000 -g UserGrid -nt 10 -r
200 -c $CATALOG_1 

4. Results

Use of the write-behind feature can lead to an improvement in performance. We ran the sample code using write-through and write-behind for a comparison in response time and database CPU utilization. Data was inserted into the cache that matched the records in the database to avoid the warm up time and produce a consistent read response time so we can compare the write response times. Listing 3 shows the responses times for the write-through scenario with reads taking less than 10 ms compared to the writes which are taking around 450-550 ms. The additional network hop and disk I/O is adding significant time to the transactions. Listing 4 shows the response times for the write-behind scenario where all transactions are committed to the grid. This leads to similar read and write response times as shown in the chart where both operations are taking 2.5-5 ms. The two charts demonstrate how the write-through scenario results in higher response times for updates whereas the write-behind scenario has update times that are nearly the same as the reads. More JVMs can be added to increase the capacity of the cache without changing the response times as there’s no longer a bottleneck with the database.

Listing 3: Chart of response times for write-through cache scenario

 

Listing 4: Chart of response times for write-behind cache scenario

 

The database CPU utilization charts illustrate the improvement in back-end load when using write-behind. Listing 5 is a graph of the database CPU usage with the write-through use case. You can see the CPU usage is continuous throughout the run with all updates being immediately written to the database. The CPU oscillates between idle state around 4% to 10-12% when it’s in use. Rather than having constant load on the back-end like the write-through scenario, the write-behind case results in low CPU with load on the back-end only when buffer interval is reached as shown in listing 6. The graph shows that the database is fairly idle at 4% CPU until the three minute interval or change count is hit and then the CPU usage increases for a short time to 12-16% when the batched changes are written to the disk. The write-behind configuration should be tuned to best match your environment with regards to the ratio of write transactions, the same record update frequency, and database update latency.

Listing 5: Chart of database CPU utilization for write-through cache scenario

Listing 6: Chart of database CPU utilization for write-behind cache scenario

Conclusion

This article reviewed the write-behind caching scenario and showed how this feature is implemented with WebSphere eXtreme Scale. The write-behind caching function reduces back-end load, decreases transaction response time, and isolates the application from back-end failure. These benefits and its simple configuration make the write-behind caching a truly powerful feature.

References

1. Gartner RAS Core Research Note G00131036, Gartner Group - http://www.gartner.com/

2. User’s Guide to WebSphere eXtreme Scale

3. WebSphere eXtreme Scale Wiki

Resources

1. WebSphere eXtreme Scale free trial download

2. WebSphere eXtreme Scale V7.0 on Amazon Elastic Compute Cloud (EC2)

Acknowledgements

Thanks to Billy Newport, Jian Tang, Thuc Nguyen, Tom Alcott, and Art Jolin for their help with this article.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

One additional disadvantage of this approach for Durability of ACID by Ibrahim Levent

Durability of ACID may be lost with this approach since an update may not be saved yet to the database if application system crashes.

Re: One additional disadvantage of this approach for Durability of ACID by Lan Vuong

This is a good point and it depends on how wide the system crash is and the replication policy selected. With WebSphere eXtreme Scale if a single JVM crashes, then the replicas on other JVMs are promoted and begin handling the changes. If multiple JVMs crash, data loss would be dependent on the number of and placement of replicas. Of course if the entire application and grid tiers crash, then durability would be lost.

Additional Questions by S Iyer

The other issues are:
Thanks for the article/data that highlights the latency gains a write-behind approach affords...Questions:
-
Do you guarantee some kind of SLA within which the values are guaranteed to be written to the database.
-
What if the Write Behind attempt fails? For transient problems - do you allow for a Retry Policy Specification and for persistent problems (perhaps an issue with the data being written behind) is there an ability to handle Poison Messages.
-
What transactional guarantees exist. i.e. since "the updates are written to the persistent store when either the specified time in seconds has passed or the number of changes in the queue map has reached the count value.", what happens if the JVM crashes during the time, the writes are being written to the DB. Once-and-only-once guarantee or Atleast-Once or No Guarantees?

Re: Additional Questions by Paulo Suzart

Good questions!
I'm particularly working with Oracle Coherence but a natural - additional - question for both IBM and Oracle solutions takes place: Rolling out new Vresions of cached objects or CacheLoaders.
We can use an neutral format for cache objects called POF (Portable Object Format) wich can be versioned and client are responsible for ser/deserialize the data. It works pretty well and avoids the fragile and slow Java Serialization, but what about versioning the classes used to store (write-behind) or retrive (read-through) data from a Data Store? Say CacheLoaders.

Major changes - in client applications or cache - may require cluster restarts, causing poor response times or unavailability during maintanance.

Wich strategies do you recommend to roll out new versions of the client application, clustered objects or CacheLoaders without breaking all the solution and most important, without stopping the cluster?

Re: Additional Questions by Billy Newport

The customer can specify an SLA in the form is at most X seconds or Y dirty entries before forcing a flush.

If a write behind transaction fails then it's written to an error queue which applications can subscribe to. Typically, it's dumped to disk and then analyzed afterwards to handle it. Write behind transactions should not fail typically though. If it's a transient issue like the database is down then it will keep retrying until the database becomes available again.

All changes buffered by write behind are replicated using the same replication policy as the application specified for the data grid. If the current server failed then all changes will be on the replica according to the replication policy.

Writes to the database are at least once also. The write behind may retry a transaction if it failed in the window between committing the database and replicating the fact that the data is now written.

Re: Additional Questions by Billy Newport

Typically, when updating a Loader (like a Coherence CacheStore) then we'd recommend a rolling update. Stop the JVMs on a box, update the Loader, restart the JVMs, wait for the cluster to stabilise (replicas to reach peer mode) then do the next box.

In summary: a synchronous solution with soft failure guarantees by Guy Pardon

Hi,

From what I understand, a power failure that includes the primary cache AND the replica will lead to lost transactions after commit - so I consider this a solution with soft guarantees wrt failures and durability (correct me if I am wrong).


Guy
Atomikos - Reliability through Atomicity

Re: In summary: a synchronous solution with soft failure guarantees by Ted Kirby

The loss of the primary shard and all its replicas can result in data loss. Note that N replicas may be specified, so you may choose your level of reliability.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

8 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT