BT

InfoQ Homepage News GRIT Protocol Enables Distributed Transactions across Multi-Database Microservices

GRIT Protocol Enables Distributed Transactions across Multi-Database Microservices

Bookmarks

At the IEEE International Conference on Data Engineering (ICDE) 2019, eBay engineers presented a paper introducing a protocol for distributed ACID transactions using multiple databases, GRIT. Support for multiple databases is key to enabling GRIT's use across microservices, which are usually implemented in different languages and may use multiple underlying databases.

GRIT aims to fill a gap in existing technologies for ACID transactions, which cannot be easily extended to the multi-database case, write eBay engineers.

In environments that involve multiple independent databases, the traditional two-phase commit (2PC) protocol was essentially the only option for distributed transactions by the system without additional application effort. However, it does not work well in a scale-out platform due to long paths of potentially many coordinating participants and the locking required over the phases. On the other hand, using a transaction log executed by a framework such as Saga will incur complex compensating logic by applications and may have business implications due to irreversible partially successful transactions.

GRIT architecture is displayed in the following picture in the context of a two-database microservice application.

As the image above shows, GRIT includes two main blocks: one group of components specific to each database —a Database Transaction Manager (DBTM), a Database Transaction Log (DBTL), and a LogPlayer for each databas — and two global components in charge of coordination —the Global Transaction Manager (GTM), and the Global Transaction Log (GTL).

A GRIT transaction occurs in three phases. During the execution of a transaction, database services collect the read-set and write-set of the transaction, without actually modifying any data. When the transaction is committed, each database submits its read-set and write-set to its DBTM, which makes a local commit decision analyzing them. All involved DBTM submit their local decisions to the GTM, which makes the global commit decision. Finally, if the transaction succeeds, the LogPlayers will send the entries collected in the DBTL to the databases, which store the data.

Overall, our approach avoids pessimistic locking during both execution and commit process and avoids waiting for physical commit. We take the optimistic approach and also make the commit process very efficient by leveraging logical commit logs and moving the physical database changes out of the commit decision process with deterministic database technology, which is similar to log play in replication.

It is worth noting GRIT can also be applied to single-database applications. In those cases, the global components are not required and the commit transaction is of minor complexity. eBay engineers show a GRIT setup meant to extend an existing database to support ACID transactions across multiple availability zones. Do no miss the original post if you are interested in the full details.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Some thoughts

    by Guy Pardon /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Looks good, but I do have some thoughts.

    First of all, the traditional 2-phase commit can scale, as we (Atomikos) have shown. So it's not per se needed to implement additional logging techniques on top of that.

    Second, optimistic locking is great but deteriorates for hot spot data.

    But it is true that microservice transactions can be a challenge. We've compiled a free ebook with our decades of experience in the matter (including distributed SOA, pre-microservices). Grab your free copy here:

    www.atomikos.com/Blog/MicroserviceTransactionPa...

  • what is the exact difference with XA-2PC?

    by Shen Haiqiang /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I don't think this protocol is different from XA-2PC protocol in fact.
    GTM is TM in XA, DBTM is RM in XA. GTM should ask all DBTMs for local commit decision and then make the global commit decision. The only difference is that there is a logical layer in GRIT, it separate physical data change to an async process.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT

Is your profile up-to-date? Please take a moment to review and update.

Note: If updating/changing your email, a validation request will be sent

Company name:
Company role:
Company size:
Country/Zone:
State/Province/Region:
You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.