BT

Java In-Memory Persistence with Space4J

| by Dio Synodinos Follow 3 Followers on Sep 20, 2008. Estimated reading time: 2 minutes |

Space4J is a simple database system that will let you work with Java Collections in memory. Since memory is several orders of magnitude faster than disk for random access to data, Space4J provides better scalability for "real-time" web applications and systems that require performance.

With Space4J Instead of having to perform a SQL SELECT to fetch a User from a database table, the developer accesses the users map (java.util.Map) and calls users.get(id). Since all data is kept in memory inside the JVM, there is no need for an extra database application, a socket connection, a JDBC driver, a SQL statement or any kind of ORM tool. Data is just there, inside objects, inside Java maps. For operations that modify data a Command object is created and then serialized and saved to disk in a log file. At restart the past commands are read from the log files and re-applied, generating the exact same data set you had for example before a system crash.

In order to prevent log files from becoming huge from time to time the application can take a snapshot of all data to disk. Space4J keeps all data inside the Space object. When taking a snapshot, the whole Space object is serialized and saved to disk. Therefore at restart only the commands since the last snapshot need to be re-applied, not all of them.  The size of the snapshot will depend on the application. Also the system has to enter read-only mode when saving the snapshot to disk, unless a Space4J cluster is used. An example of such a deployment would be a web application in load balance, where every web server would have its own Space4J node from a cluster

Space4J comes with a complete indexing framework that supports four different types of indexes to search data in a variety of ways. Also it can be used alongside a regular database  for offline work, data  warehousing, reports, etc.

Space4J uses the Java 1.6 concurrent data structures for concurrent read/write access to data so writers only block writers, readers don't block or get blocked by anything. This means that modifications are done one at a time while read-access operations are done concurrently without any ConcurrentModification exceptions!

 You can download the latest release (0.9.1) or browse through the source repository.

For a more information regarding the emerging paradigm of shifting data access from disk to memory for performance and other data access issues, you can read “RAM is the new disk...” by Steven Robbins, here at InfoQ.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Prevayler by Peter Monks

How does Space4J compare to Prevayler?

Re: Prevayler by Ladislav Thon

I wanted to ask very same question, but it is apparently answered on the product site:



How Space4J compares to Prevalyer? Space4J and Prevayler are two free Java implementations of the same concept. Prevayler has the merit of being the first implementation created from scratch by Klaus Wuestefeld. Klaus also has the merit of pushing the idea of a prevalent system that is both possible and desirable in many cases. Although Space4J and Prevalyer are centered in the same idea, they have totally different APIs and implementations. It is pretty much like Struts and JSF for web frameworks.

Broken? by Martin Probst

I wonder if this is really worth the hassle. You will get into terrible problems once your data exceeds some GB, concurrency is difficult to get right for access over several collections (no transactions), and there is no rollback or similar.

I think it's questionable if for such a limited approach if it's not easier and more straight forward to write your own simple write-ahead logging persistence, in particular if you have high performance requirements. And in other cases, just use a database.

My 2 cents by Thai Dang Vu

If you don't use SQL statements then it'll be difficult to switch to real database when we need more space than the memory can provide.

Caching by Gleb Liferenko

Such a technology should be used in combination with, not instead of a relational database, similar to how [[cached]] is used. Also, it would solve your transaction problems.

Re: Broken? by Sergio Oliveira

Martin said: concurrency is difficult to get right for access over several collections (no transactions)



This difficult task was done for me by Doug Lee in the java.util.concurrent package. Like it is said in the article: "Space4J uses the Java 1.6 concurrent data structures for concurrent read/write access to data so writers only block writers, readers don't block or get blocked by anything. This means that modifications are done one at a time while read-access operations are done concurrently without any ConcurrentModification exceptions!"



Martin said: there is no rollback or similar



This is a drawback I agree. But commands should be atomic by nature and you can always check for errors before modifying. That's possible because every command is executed in isolation from the other ones, in other words, writes are serialized and never executed concurrently.



Gleb said: Such a technology should be used in combination with, not instead of a relational database, similar to how [[cached]] is used. Also, it would solve your transaction problems.

Yes, you got the correct idea!

GC by Dax Abraham

What about garbage collection when the size of the object in memory becomes really huge ...

Re: GC by Sergio Oliveira

You must be talking about the Space object that holds all other collections. This guy is never GC. There is no need to.

You should think as Space4J as a relational database, but:

Database = Space object
Table = Java collection
Index = Java Map
Selects = direct collection access like map.get(id)
Inserts/Updates/Deleted = Space4J command

How is this different? by Sarath P

From Object Caching?
From memcached?

Re: How is this different? by Sarath P

oops.. I lost context! ignore my earlier question

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

10 Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT