Terracotta open sources JVM clustering
OpenTerracotta’s DSO (distributed shared objects) allows the transparent clustering of user-defined classes, Java primitives, and even core Java class libraries, including its concurrency abstractions. DSO is a hub and spoke architecture, with a central server replicating data across JVMs as needed. Objects retain identity across a cluster, and the central data store (which itself can be replicable) ensures smooth failover. Terracotta replicates changes from one heap to heaps in other VM's. Terracotta also coordinates synchronization between threads, and can migrate threads across server instances. Bytecode instrumentation of classes declared clusterable is also used, resulting in fine-grained replication field-level changes across the cluster, without the need to declare objects as serializeable.
Terracotta for Spring is a customization of DSO which allows the transparent clustering of Spring Application Contexts, Singleton bean types, Application context events, as well as Spring’s JMX support.
Terracotta Sessions is also a customization of DSO that can cluster HTTPSession objects, supporting Apache Tomcat, BEA WebLogic and Websphere CE. Being based on DSO, Terracotta sessions was proven faster than other open source session clustering alternatives due to it's ability to distribute only field-level changes and also push changes only to servers that needed them.
Terracotta can also be used as a Grid solution, implementing the Master/Worker pattern defined in the CommonJ spec from IBM and BEA.
InfoQ spoke to Terracotta founder and CTO Ari Zilka to learn more about how and why this VC-backed startup went open source.
InfoQ asked how Ari was able to convince the investors to make such a drastic change to Terracotta's business model. Ari explained that Terracotta had board members supporting both sides, but that two datapoints easly aligned the board. Firstly, 80% of installs of Terracotta were in environments that were mixed with open source, usually Spring, Struts, and Tomcat. Second, many companies told Terracotta that the process of going through a proof of concept and evaluation is too time consuming from both a practical and legal standpoint, but if Terracotta were already out as open source then we "would have already been using you, running in production and we would have been calling you for support."
In terms of licensing, Terracotta created the Terracotta Public License which extends the Mozilla Public License with an attribution clause requiring companies shipping with Terracotta to include the equivalent of a 'powered by' logo. Ari explalined:
We wanted MPL, we needed a license that was a house hold name. We liked GPL from a business safety perspective but the community seems against it because of its viral nature... we wanted people to be able to adopt the software, both ISV and end user, minimize forking and get improvements back into the product, and attribution ensures that direct competitors have to give us credit which would give us more protection from forking.Comparing License contract revenues with support contract revenues, Ari explained that all the legalize associated with buying commercial licenses, idemnities and escrow issues is daunting, whereas with a subscription support model:
You pay a certain amount per year, and that amount is much less than a license so you don't have customers asking for a 90% discount... Subscription means that you can hold to your prices as long as they are commensurate with your costs... No one asks for source code escrow for example because it's open source. We don't need to define trigger provisions that give you rights to Terracotta's source code and I don't need to protect it because it's already open... Our support offering would make us more money after about 5 years than a licnense fee upfront but we have to earn the customers subscriptions year on year and provide value.On how he would position Terracotta to other open source clustering solutions, Ari explained that for small to medium companies, Terracotta gives you the developer benefits, the operator (datacenter) benefits and lower cost of ownership:
The core value proposition for Terracotta includes performance, ease of development, and ease of operations. Object identity works for clustering at heap level which means you're basically building to stateful model, which nobody else can say. For example, if you' were using serialization you would have to do things differently as a result of clustering than you would have done in a prototype of the same application. For ease of development thus I think we win. For ease of operations we are as good as anything else out there. We have a central server, which can be managed from one console, but peer to peer technologies can do that with JMX broadcast and such. And then our server clusters so there is no single point of failure so we're similar to other solutions. I would position it to an operator that you get the benefits of stateless while developers on your team get the benefits of stateful and we are the only solution that has both... And then from a scalability perspective I think we are one of the youngest companies out there and there are more scalable solutions from us, but we are scalable enough for small to medium companies.Moving onto more technical topics, Ari explained how Terracotta is typically being used:
Developers use Terracotta to share objects across JVMs (notice, we didn’t say "data" because sharing objects requires cluster-wide object identity and Terracotta’s customers rely on that). Specifically, developers write applications that contain more state in-Heap. If any one node fails, all other JVMs have access to the same data so nothing is lost; sort of a stateful model in a stateless runtime environment. Also, clustered apps scale well with our fine-grained replication (field-level) and fine-grained locking (object-level).Asked if people are actually clustering Java primitives and using standard Java concurrency features in clustered fashion, Ari gave some examples:
Clustered apps that were previously leveraging JMS and RMI to do things like scatter-gather / MapReduce / Master-worker and the like, are moving to a pure POJO model with LinkedBlockingQueues and wait() / notify() that clusters as part of the runtime. They switch because their current architecture is too intrusive or hard to maintain or not scaling as a function of nodes/size of clustered data.While Ari was careful to position Terracotta as a no-brainer answer for small to medium companies, he also explained how one customer was using Terracotta in a large-scale usecase:
Large can be defined as a large number of JVMs, large amounts of clustered data, or a significant rate of change to objects. Under one scenario, we encountered all three with thousands of JVMs and upwards of 80,000 transactions per second. The customer helped us design a solution where their application split objects across Terracotta servers by hashing the objects into separate hashmaps, each of which was accessible by all JVMs. In each map the app stored one-nth of the objects in the system. Then, we tuned Terracotta to not prefetch / batch objects and, furthermore, not to cache in the local JVM. There was no load balancer required because Locality of Reference was set to the Terracotta Servers. And, there were 10 of them. Each one was responsible for 8,000 transactions per second. This worked very well for the customer and outran many other options they had tried. While they had to write code to get the app to scale, the objects were still POJO and the customer was satisfied that they could continue to scale as their application grew.On potential misuses of Terracotta:
Terracotta is location-transparent meaning locality of reference is not maintained. State can be faulted-in from the Terracotta server as needed. However crossing JVMs implies that the calling thread must incur network overhead and latency costs. So it is recommended that applications maintain Locality of Reference as much as possible.On how Terracotta ensures fault tolerance considering its central coordinating hub, Ari explained:
In persistence mode, Terracotta allows state to be persisted across JVM restarts. However, it is not a system-of-record and lacks database features such as recovery, backup etc. So use it as a persistent data-store carefully only in situations where occasional data-loss in edge-cases might be allowable.
In our operational past w/ several large Internet sites, hub-and-spoke lent itself well to infrastructure services; operators know how to make machines fault tolerant and services highly available. Terracotta’s Server leverages this now-common-place Active/Hot-Standby architecture. Your application’s JVMs transparently connect to the standby Terracotta Server upon failure of the active Terracotta server. The design avoids split-brain / network partitioning issues. Honestly, we will have to start clustering Terracotta servers in a peer-to-peer fashion once a use case outstrips the capacity of a single active Terracotta instance. For now though, the fact that we push as much data as changes in your objects means that even the largest installations have not outgrown a single server instance.Finally, on how Terracotta supports data partitioning architectures (co-locating data access with data residency to avoid the latency of inter-node data-sharing), Ari explained:
Terracotta has implemented the Master/Worker pattern.It is intended to help with Locality of Reference by moving the app to the data instead of the data to the app, but it requires uses POJOs to function. Our customers call this many things from “grid” to “Master / Worker” to MapReduce to scatter / gather. Conceptually, work gets input to a queue and is processed by multiple, collaborating worker-JVMs, each of which owns some subset of the data. The pattern provides all the required plumbing to route data to workers and keep it there (off of other workers and off the network).An Enterprise edition will also be available for purchase, but it is identical to open, except that there is a contract with indemnification and perpetual license to a particular version of TC's product.
In terms of community integration with Terracotta, Ari told InfoQ that Tomcat support company Covalent will be bundling Terracotta in their Tomcat distribution. The Geronimo team is considering bundling Terracotta in the Websphere CE as a clustering option. Terracotta will be linked to as clustering option on the Tomcat clustering FAQ and by Glassfish. Apache Tomcat and Tribes committer Filip Hanik is also a committer on openTerracotta, as is Jeff Genender, Geronimo committer.
Great decission guys!
:Architect of InfoQ.com:
.w( the_mindstorm )p.
Co-founder of InfoQ.com
Not new, but good
Hey Floyd, if you remember, I wrote the review for Gemstone way back when on TSS :-)
great decision. Another +1 for the java community. Sun's open sourcing of java and this announcement makes(keeps) Java THE STANDARD for large web applications.
As Tim Bray said, because there are so many options/models for deployment in java world, (ordinary) people don't do a good job of scaling the apps. If Tomcat can be bundled together with Spring and Terracotta and a good IDE is available for the developer to develop and debug for this combined platform, then Java's position should improve as far as combined DEVELOPER productivity and app runtime performance.
I hope that openTerracotta will become the place for bundling Tomcat + Terracotta rather than Covalent. I don't have anything against Covalent but more developers will get a chance to use Terracotta if there is an easy way to have them bundled. Covalent can certify their own copy of the bundle but it will be nice if it(a bundle) is available for all developers.
Again, since a lot of developers are looking at spring, does it not make sense to bundle Tomcat, Spring and Terracotta at terracotta.org ? I hope the eclipse plugin can integrate with the Spring IDE to make developers task easier than today.
some things to look forward to :
1) Tomcat + Terracotta bundle
2) Tomcat + Spring Terracotta bundle
3) Tomcat + Spring Terracotta + IDE bundle
4) Tomcat + Spring Terracotta + IDE + Linux/Solaris LIVE CD bundle
5) Tomcat + Spring Terracotta + IDE + Linux/Solaris VMWARE player
6) Tomcat + Spring Terracotta + IDE + CentOS (Linux) distro
7) AT THE LEAST, a screencast(en.wikipedia.org/wiki/Screencast, www.rubyonrails.org/screencasts)
in "HOW TO BUNDLE tomcat and Terracotta for Java Petstore app"
With this development, it will be nice to know how coherence(the leading java clustering app) will react to this.
It will be nice if the Geronimo, JBoss and Glassfish community picks this up and adds it to their distros. Bonus points, if the distros include help with development and debugging.
These J2EE distros can bundle a sample app : Java Petstore + Terracotta ?
Re: Not new, but good
Hey Floyd, if you remember, I wrote the review for Gemstone way back when on TSS :-)I remember. :) Infact, TSS hasn't updated their reviews winners on the reviews rightbar since 2003 so your name is still there :)
Does anyone understand what the attribution section of the TPL means? Must all products and solutions built with Terracotta display the powered-by text, or just OEM usage of Terracotta? Is the clause transitive (i.e., if ACME Corp were to ship Terracotta, would the attribution clause follow through to ACME Corp's clients)? Are there any provisions / expectations for localization of the text?
Re: Attribution clause?
Thanks for the feedback -- great question.
The attribution clause of the TPL says that if you distribute any "covered code" (defined as original Terracotta code, or modifications, or the combination of the original code and modifications), then you have to put a "Powered by Terracotta" text notice at the bottom of each GUI that appears in the product/solution that you build with Terracotta.
So, to address your example, if the ACME Corp were to ship Terracotta, then ACME Corp would need to comply with the attribution clause. If ACME Corp's customer consequently built a new product and shipped it, then it, too, would need to comply with the attribution clause. However, if ACME Corp only used Terracotta in-house and didn't distribute any "covered code" to anyone outside its company, then the attribution requirement would not apply.
Hope that helps, and thanks for the feedback. We'd like to include your question on our licensing FAQ, too.
Re: Attribution clause?
Also, for your localization question, no - there aren't requirements in the TPL about localizing the text. "Powered by Terracotta" is fine.
With this development, it will be nice to know how coherence(the leading java clustering app) will react to this.
From my comment on TSS:
This is a wise move for Terracotta, and good luck to them. And maybe this will make it easier for them to add support for Coherence now ;-)
Tangosol Coherence: The Java Data Grid
I'm new to both Tangosol and Terracotta and a little confused about where they overlap and where they differentiate. Would you be able to illustrate where and how Tangosol's technology offerings would be able to leverage Terracotta's offerings and the synergy they could create together? I'm sure it would be really helpful to end users who have just heard about and started looking into both technologies when this news surfaced (such as myself).
Terracotta DSO uses a TCP/IP-based client/server architecture that consists of client-side instrumentation (byte code changes for "transparent clustering"), combined with a central server ("hub") for sharing state between application servers. In a Coherence cluster, the analogous components would be our free Coherence Data Client on the application servers, combined with any of our server-side editions (e.g. Coherence Caching Edition) as a fault-tolerant scale-out solution for state and data management.
Terracotta explains their clustering as follows:
Terracotta servers can be deployed as an active-Primary plus a passive-Secondary (i.e. 1 or numerous hot-standby(s)) - the hot-standby shares disk with the primary Terracotta Server. On failure of the primary, client-JVMs transparently connect to the standby.
Any my response:
For reference, this is one of the primary differences betweent Terracotta clustering and Tangosol clustering. Terracotta can cluster two servers (one hot, one standby) while Tangosol can run a couple thousand servers as "hot" (active + active + active + etc) in an n-way fully-connected mesh (virtual channels). Our server throughput in a 100-server system is 50x that of a hot+hot 2-server system, and (in a fully switched fabric) our throughput in a 1000-server system is 10x that of a 100-server system. And failover time (automatic, without data loss or interruption of application flow) is still typically sub-second.
Regarding the connections between the application servers and the Terracotta server, it is a TCP/IP client/server connection (no fundamental differences at the wire level from a Telnet session, JDBC connection or RMI). It is analogous to our free Data Client or our low-cost Real-Time Client.
Speaking for me personally, I would evaluate plugging Terracotta in through the TCP/IP connectivity (ours, theirs, whatever), because Terracotta has focused on the programming model (AOP, Spring, etc.), and it could be a good addition for working with data in a Data Grid.
Tangosol Coherence: The Java Data Grid
Thanks and a few opinions to add to the mix
Thanks so much for the kind words. I am very excited about our having come this far so quickly. We officially launched the company at J1, 6 months ago now and since then, OSS has been the focus. Very exciting and I am looking forward to all the new and great things the community will help us head towards in the next few months. A couple of kewl things that are coming down the pike:
1. Active / Active Terracotta servers
2. Wide Area asynch replication
3. more transparency (always more transparency. I hear one of our engineers has a mockup of DSO-ized threads migrating around the cluster that I want to wrap my head around)
4. Continued tuning. Although we have succeeded recently at scaling for a use case under EVERY PAGE REQUEST for one of the top 10 websites in the world, we have to do more.
5. More OSS bundling announcements are forthcoming
6. More tools for our community and partners around hosting their own clustering projects and use cases.
Generaly speaking, I want to make sure people's expectations are set correctly. Terracotta can scale for most applications because it is fine-grained. What I mean is only the bytes that change in your objects get sent to us, and thus our throughput is extremely high. Some of our most satisfied customers are so because they were able to achieve scale with simplicity. That said, don't focus on drop-in...focus on impact to the development model. Whether or not we drop in in a particular use case, we have found that our technology helps applications become more stable and more scalable because our runtime tooling helps people find bottlenecks they never knew they had. The most successful use of Terracotta is in returning control of state back into the hands of the Java developer. Clustering HTTP sessions, singletons, (and factory-pattern internals), clustering certain Spring Beans, etc. As Rod Johnson recently put it, "business data is that which is needed for audit purposes and / or is queried by the business via a Data Warehouse." This is actually a quite eloquent definition if you think about it. Terracotta affords a developer control over state in a non-repository fashion and use cases that rely on "durable / clusterable" heap--meaning they just leave POJOs as such--will find the most benefit today.
Transparent JVM clustering solutions will improve
One can argue about how much scale-up work the Terracotta hub needs today, but it's hard to dispute Ari's basic premise: that (1) the JVM can transparently replicate objects by watching the GET_FIELD and PUT_FIELD bytecodes, and (2) that you can insure thread safety across a cluster the same way you would in a single VM, as long as you can meet the contract of the MONITOR_ENTRY and MONITOR_EXIT bytecodes.
If, in 5-10 years, it becomes as automatic as garbage collection is today, that is a huge ideological victory for Ari, Terracotta, and the KISS principle. I think they're on to something very big.
For example, from Martin Fowler OOPSLA2005:
"While I'm on the topic of concurrency I should mention my far too brief chat with Doug Lea. He commented that multi-threaded Java these days far outperforms C, due to the memory management and a garbage collector. If I recall correctly he said 'only 12 times faster than C means you haven't started optimizing'."