How would a modern cloud platform look if we could retain the cumulative knowledge we have as an industry today, yet start all over with new technologies, unencumbered by legacy technologies and protocols? And what if that cloud platform would be able to support the next generation of the internet - the Internet of Everything (IoE)?
At Jenkov Aps and Worpcloud Ltd. those are exactly the questions that motivated us to partner in our new cloud project VStack.co early last year. Throughout 2015 we analyzed most parts of modern application stacks, from the high level architectural aspects, to concrete technologies like databases, query languages, message queues, backup solutions, distributed computing models, even network protocols.
The analysis and proof-of-concept implementation work is an ongoing effort, but at this point we can see that the resulting technology stack will look significantly different from existing cloud platforms. In fact, we find that several of today’s internet technologies could be replaced with better alternatives.
Whether the results of our project will live up to our expectations is too early to say because the platform is not yet fully implemented. We have naturally had to change designs several times as we learned more, but, the designs are now stable enough to start discussing them with the developer community.
We could not describe the whole architecture in a single article, nor would it make sense to discuss it all, since parts are only visible internally. Therefore this article focuses on just one central aspect of our architecture, IAP, our suggested replacement for HTTP.
IAP - Internet Application Protocol
Internet Application Protocol (IAP) is intended to be a general purpose network protocol for web 3.0. We believe that standardizing on an appropriate internet application protocol could have similar ramifications for applications and devices that USB had for the PC and peripheral hardware, that internet connected applications and devices can interact simply by connecting them to the internet.
IAP is not yet fully specified, but we know pretty much how the core of IAP should work. The current version of the IAP specification is available here.
IAP addresses many of the use cases that HTTP 1.1 ignores. While HTTP2 and WebSockets definitely address several problems not addressed by HTTP 1.1, we believe that more is still required, as we describe in our blog: Why HTTP2 and Websockets are not Enough.
IAP is a free flow message based protocol. The communicating nodes exchange messages, just like HTTP requests and responses. However, IAP does not require that each message have a response. Being a free flow protocol, IAP specifies only that nodes exchange messages. Messages can flow freely in both directions of a network connection as the communicating nodes see fit for the purpose of the communication. We have described some of the core message flows in more detail in our tutorial IAP Message Flows.
In order to move IAP forward towards completion, we felt it was time to introduce the discussion to the community about a replacement for HTTP, and to put forward a suggested solution. Actually, this article will focus mostly on ION, the binary object notation used in IAP. We have chosen this focus for three reasons:
First, the ION specification is much closer to stable than the whole IAP specification. Therefore it makes more sense to start the discussion with ION.
Second, ION can be used independently of IAP, meaning you can use ION with HTTP as a more compact and better performing alternative to HTTP / JSON, for all non-browser communication. Examples would include use cases such as a mobile app communicating with your backend, or backend-to-backend communication.
Finally, ION has an impact on what IAP can be used for. So even before the IAP specification is completed, understanding ION and the basic IAP message structure will give you a reasonably good idea about what IAP will look like in the end.
Introduction to ION
As we mentioned IAP is a message based network protocol. All IAP messages are encoded in a binary format called ION; short for IAP Object Notation. A binary representation results in more compact messages and higher parsing speeds than textual data formats like XML and JSON.
ION is a TLV (Type Length Value) format. Each ION field contains its type, length and value in that order. We have a more detailed specification of the ION encoding here.
The encoding of ION is similar to the encodings of binary representation formats CBOR and MessagePack, but ION deviates from these formats in a few key aspects. First of all, both CBOR and MessagePack were originally designed to be binary representations of JSON with added ability to carry raw binary data (e.g. files).
ION is designed to be a general purpose data format, designed for modeling the following common data structures:
- Raw bytes (files)
- Primitive binary values (boolean, integers, floating points, UTF-8 strings, UTC date time).
- Streams of independent values (unbounded)
- Arrays of independent values (bounded)
- Maps (key, value pairs)
- Objects
- Object graphs with circular references
- Tables
These structures can be nested inside one another; so for instance, you can model a compact object graph by nesting tables inside tables. These basic data structures can model both JSON, XML, CSV or raw binary data, and you can actually convert JSON, XML or CSV to ION and back again.
Tables
The most significant difference between ION and MessagePack / CBOR is the compact representation of tabular data. Some examples include:
- arrays of objects of the same type
- database tables
- CSV files.
Being able to represent tabular data compactly delivers significant benefits in performance, bandwidth and memory usage. Our measurements show that data sizes of ION tables are often 50-80% smaller than the same data serialized to an array of JSON objects. Similarly, ION tables are typically 25-50% smaller than the corresponding MessagePack or CBOR encoded data. Many services send and receive lists of results over a network, so compaction can profoundly impact performance in such common use cases.
Read and write speeds of ION tables are up to 2.75 times higher than the corresponding read and write speed for Jackson / CBOR, and up to 5 times higher than the corresponding read and write speed of Jackson / JSON.
Flexible Objects
Another difference from ION to JSON, MessagePack and CBOR is that ION objects can contain any arbitrary sequence of ION fields that an application sees fit. An ION object, like JSON, can contain property name value pairs, but it can also contain a mixed sequence of other fields. For instance, an ION object might contain a sequence of UTF-8 fields and ION object fields to mimic XML text nodes mixed with XML elements.
ION objects can be restricted to property values only. This results in very compact representations of single objects, and can achieve read and write speeds that match Google Protocol Buffers (faster writes, slower reads), but with the advantages of being schema free without the data size caveats attributed to Protobufs (Protobuf acknowledges that it is not a good encoding for raw bytes such as files.)
Using an ION object that contains only property values requires you to know the sequence of the fields inside the object. This might not be a good idea for public APIs, but for internal services that are tightly coupled and require high performance, having this option is useful.
Arbitrary Hierarchical Navigation
The benchmarks mentioned earlier measured the speed of reading ION into Java objects, and writing Java objects into ION. However for maximum speed, instead of parsing ION into Java objects, we recommend directly consuming the data in its binary form; ION is designed with this usage model in mind.
All ION fields contain the length in bytes of the field value. First, knowing the length in bytes of an ION field makes it easier and faster to extract the value from the binary data. You do not have to examine (parse) every byte of the value to see where it ends, like you do with textual formats like JSON and XML. Second, knowing the length in bytes of a field value makes it easier and faster to skip over any fields that you might not want to process.
Any ION fields that can contain other ION fields (such as Object, Table, or Array) also contain the length in bytes of their value. This makes it easier and faster to skip over a whole “branch” of an object graph without having to parse the fields inside it. This is what we mean by “arbitrary hierarchical navigation”: you can quickly and easily navigate in and out of graph-like data structures.
Arbitrary hierarchical navigation is one area where ION offers a significant improvement over MessagePack and CBOR, for only a slight difference in encoding. In MessagePack, elements that can contain nested elements list the number of elements inside the containing element - not the number of bytes. So to skip over an element with nested elements in MessagePack or CBOR you need to iterate over all of the nested elements to find where the containing element ends.
We have created a simple read-and-use benchmark to show the speed difference between parsing ION data into objects before using it and using the ION data directly in its binary form. This benchmark reads a table of 10 objects, each with one property of type long, and calculates the sum.
The first version of the benchmark reads the ION data into a Java object graph (via our Java reflection API) before summing up the long properties. The second version of the benchmark sums the long properties directly from the raw ION data.
Summing the long properties directly from the raw ION data is around 10 times faster than first parsing it into Java objects with our reflection based API. That is approximately three times faster than using Google Protocol Buffers (which also parses Protobuf messages into Java objects), and approximately 15 to 40 times faster than parsing JSON to Java objects and using the objects (using Jackson’s reflection based API).
Additionally, the benchmark uses all of the ION data graph. If a calculation uses just parts of an object graph, the performance improvement between parsing the binary data into objects first and using the binary data directly might be even more dramatic.
ION As File Format
ION is designed as a data format for network messages, but can also be used as a file format. At VStack.co we use ION for both data files and log files. Having a single data format makes it easier for us to work with data. Network messages can be written to disk for later analysis or later replay, and data from files can easily be included in network messages.
More Information About ION vs. Other Data Formats
We have more details of the serialized length and read / write performance of ION in our performance benchmarks text.
We also have more details about how ION compares to other data formats in our text “ION vs. Other Formats”.
IAP Message Structure
An IAP message is encoded as a single ION object field. Two nodes communicating via IAP exchange ION object fields. ION object fields contain nested ION fields inside of them. The nested fields make up the message headers and message body.
Because IAP messages are ION object fields, a server receiving an IAP message knows within at most the first 16 bytes of the message, and generally in the first four to five bytes, exactly how long the full message will be. This makes it easy for servers to allocate the correct amount of memory for small messages, enabling servers receiving large numbers of small messages, to better utilize their memory.
That each IAP message is a single ION object field also makes it easy to track when a full message has been received. Once you have received the number of bytes that the ION object field declared, you know a full message has been received.
Our current IAP server implementation can echo (ping-pong) around 200K (36 byte) messages per second with a single-threaded server running on one core, with four connected clients running on the same physical server as the IAP server. The server is a commodity hardware box, quad core CPU, Haswell architecture, DDR3 RAM. A client sends in a message and the server sends the same message back. When the client receives the echo from the server it sends the next message, the server echoes it etc.
With clients pipelining the messages and the server running on all 4 cores instead of one, we should be able to cross 1 million messages / second.
Since ION object fields are compact and easy to parse, ION is also well suited for small devices with limited CPU and memory resources. This makes IAP suitable as a protocol not only for typical web applications and backend services, but for the Internet of Things.
IAP Semantic Protocols
Trying to define a single network protocol to serve all current and future use cases is hard if not impossible. Instead of attempting that, IAP is designed to consist of multiple smaller protocols that can be used in conjunction. These protocols are divided into a set of core protocols and a set of semantic protocols.
The core protocols specify how generic functionality like acknowledging received messages, caching etc. will work. This is functionality that can be useful inside a wide range of network protocols, providing cross-protocol functionality.
The semantic protocols address concrete use cases like file exchange, streaming, chat, VoIP etc. IAP will define a set of standard semantic protocols but also provides flexibility for you to define your own, custom semantic protocols.
All core and semantic protocols will use the same basic message structure. Many semantic protocols will also use similar communication patterns. That means that it should be possible to implement a single message oriented server platform that can be used to serve most, if not all, of the core and semantic protocols.
The core IAP protocols and semantic protocols are not finalized yet, but the big design decisions have been made, and we will announce the core and semantic protocols as they stabilize.
IAP Transport Protocols
Since one message exchange pattern or one protocol can't fit every use case, neither will one underlying transport protocol fit all. Therefore IAP is designed to be able to run on top of either TCP or UDP. Exactly how that will look is also not finalized yet, but we are getting there.
Common Reactions to IAP
This is not the first time we have have discussed IAP with developers. In the following sections are some of the reactions we have received and our responses.
A binary protocol is hard to debug
That is true. But, first of all, most databases use proprietary, binary protocols to transport data between the database server and the client API, and we haven't heard many complaints about that.
Second, ION can be converted to XML and back without loss of data. That means that you can convert ION messages to XML and view them in a text editor. It is even possible to edit the XML file and convert it back to ION, for use in unit tests, debugging etc. Conversion between ION and XML is not yet implemented, but we expect to get around to it sometime in Q1.
Compactness does not matter when you ZIP compress the data.
A common argument against compact data is that you can just use ZIP compression on the wire. However, because of the CRIME and BREACH attacks it is currently recommended not to use data compression over encrypted connections (TLS / SSL). In this perspective ION’s compact table encoding can make a significant difference.
Even if all those JSON bytes were ZIP compressed on the wire, they would still need to be decompressed and parsed, which takes CPU time. ZIP compression may reduce on-the-wire transport time, but it will increase parsing time. Even if the increase is small, it is still unnecessary.
IAP and ION is not mature / production ready
It is true that IAP and ION are not as widely used or proven as for example HTTP or JSON. Nonetheless, we have decided to base our platform on IAP instead of HTTP; all data entering and exiting the core VStack.co platform will do so via IAP. If IAP / ION has problems, we will be among the first to know.
One size won’t fit all
No - that is why we designed IAP to have a very simple core message structure extendable to fit different semantic protocols.
Have you looked at / compared to format XYZ?
We have heard this question a lot. We have compared ION to JSON, MessagePack, CBOR and Protobuf. Furthermore, we have studied Cap’n Proto, Avro and Thrift. The encodings are not that different, so speed should be comparable, and depend more on the implementation than the format.
However, one major decision we made that makes ION different from e.g. Protobuf and Cap’n Proto is that ION should be self describing, meaning you can make sense from an ION file without having a separate schema. This is necessary to make ION messages routable by intermediate nodes which do not know the schema of the message.
Being self describing makes ION messages slightly bigger and slightly slower to parse, but it makes ION easier to work with. You can make sense of data and log files without knowing the exact format the log messages are written in. You don’t have to be afraid that you will lose the schema, or that you have store the schema in the beginning of the file to avoid losing it. You can even convert an ION message to a textual format and back again without knowing its schema.
Furthermore, if you need to further increase transfer speeds you can leave out a lot of ION’s meta data so your messages get close in size to data formats that require a schema (e.g. Protobuf). Even so, you can still make some sense of the data in an ION message, as each field is still explicitly marked in the binary data.
IAP Tools
We have an open source API for working with IAP and ION called IAP Tools. Currently we only have a Java implementation but we expect to develop IAP APIs for D and C# too. If these prove successful we might consider porting to other languages too.
You can find IAP Tools for Java here.
The code is not yet production ready, but it is good enough to play with ION and make simple proof-of-concept implementations for your system. We expect the ION related part of the code to mature during Q1. The IAP related part will take somewhat longer.
Another toolkit that is considering supporting ION is QBit (lead by Rick Hightower), a high speed microservice toolkit. QBit services communicate internally via messages. Today, QBit services often use JSON as message format. Using ION instead of JSON would provide smaller messages, higher read / write speeds and more flexibility in choice of data structures.
We want your feedback
We are very interested in hearing what you have to say about a new network protocol. We have heard many developers complain about HTTP; please let us know why, so we can make sure IAP solves your problems.
A new network protocol will mean a lot of new software will need to be written, but it will also open up a lot of new possibilities - especially if we can settle on a more versatile network protocol than HTTP.
About the Author
