AxonDB is a new data storage purpose-built for event sourcing with support for transactions and pushed-based event publishing that recently was released by AxonIQ, the company behind the Axon Framework. To make sure performance is constant, irrespective of storage size, the architecture is specifically targeting reading data — according to AxonIQ the performance is stable even with huge amounts of events.
AxonDB offers a fully ACID compliant transactional model meaning that multiple events can be appended in a single transaction. Consistency is achieved by requiring that all events for an aggregate are stored with a unique, contiguous and increasing sequence number. To reach durability with high throughput, it's recommended to run in a clustered setup.
When an event is appended to AxonDB, it will store this event but also push the event to listeners and event processors that are running, thus removing the need for regular polling and the latency that it brings.
Other characteristics and features of AxonDB include:
- Built from scratch in Java
- Manages files directly, with no underlying database system
- Is append-only by design
- Has built-in support for snapshots
- Support for ad-hoc queries, including using XPath for querying — this can be used for debugging, support or similar needs
- Support for clustering based on floating single-master with write quorum
- Includes support for migration from a relational database or MongoDB
In a presentation about the requirements for an event store and why they chose to build AxonDB, Frans van Buul, Commercial Director at AxonIQ, emphasized that according to Gartner, Event-Driven Architecture (EDA) is one of the top 10 strategic technology trends for 2018, and noted that event sourcing is a specific type of EDA.
The reason for designing a new event storage was that the team didn't find any existing database supporting the specific requirements of an event store, which included:
- Support for reading all events for an aggregate, and all events since a specific point in time
- Read back events in write order
- Atomicity to allow for appending multiple events at once, and reading only of committed events
- Constant performance irrespective of storage size. Snapshotting and optimization for recent events are important in large event stores to overcome performance issues
- Support only for the appending of events. There is no need for insert of events at a random point, and no update or delete of events since they are an immutable fact
- Publishing of stored events to event listeners
RDBMS is a well-established technology with good support for atomicity. The big downside for van Buul is performance. Because of the architecture, the performance cannot be maintained with a huge number of events.
Document databases like MongoDB has the capability to store large volumes of data, but van Buul sees a problem with the transaction boundary that covers one document only. A solution is to store multiple events in one document, but this raises other problems.
Kafka is message-focused and is extremely scalable regarding total number of messages. It's mainly designed for temporary storage but can also store messages indefinitely. The main problem occurs when reading all events for an aggregate. Van Buul notes that you can use topics but that doesn't scale to millions of aggregates.
Event Store, with Greg Young as lead architect, is another storage persistence engine designed specifically for event sourcing. It places heavy emphasis on projection logic inside the event store, but van Buul prefers to keep this logic outside of the storage.
Due to the problems they saw in existing data storages, they eventually decided to build a new event store, purpose-built for event sourcing. This resulted in the now released AxonDB — a commercial product available in four editions, including a free developer edition that also includes the documentation.
In the upcoming MongoDB 4.0 multi-document transactions will be included.
A GDPR module that can work with an immutable list of events is available from AxonIQ. It ensures that personal data within events is stored in an encrypted form by using encryption keys unique to an aggregate. Removing personal data is done by deleting the key — no personal data in the aggregate can then be read and this is done without changing any existing events.
A one-day conference day about the Axon products is scheduled for September 21, 2018