Protocol Design and Implementation with Martin Thompson
Architects and developers routinely miss opportunities to introduce stability into their systems by not addressing the need for defining interaction protocols within their domains. Recently, InfoQ sent Martin Thompson, an expert in high-performance computing, a series of questions about protocol design and his project Simple Binary Encoding (SBE).
InfoQ: As far as computer science and systems go, what is a protocol?
A protocol is an agreed set of interactions which are governed by rules. A simple example that most people use all the time is a file. The protocol for interacting to a file is an open action, followed by zero or more read/write actions, then followed by a close action. The protocol does not just define the actions that can be performed, more importantly it defines a set of rules that govern the possible valid orderings for these action. In the file example it is not possible to read, write, or close a file that has not yet been opened.
In systems design we often hear people talking about APIs which is an incomplete picture of how one should use a system. If more people published protocols for interacting with their APIs then systems would be more usable.
Unfortunately our industry suffers from very poor use of terminology. Given the immaturity of our industry this is not a surprise. Maybe we could consider that we are living in the age of computer alchemy and within time we will mature to better practices and understanding. So often people misuse the term "protocol" to mean other things, often tangential things. For example, the popular Google Protocol Buffers (GBP) is actually a codec but many call it a protocol. We also see this when XML is called a protocol yet it is being use as a grammar or as an encoding. Message sequences governed by ordering rules to make a protocol are encoded into ASCII or binary formats by a codec. GBP and SBE are codecs. SMTP is a set of messages organised into a protocol for interacting with a mail server.
InfoQ: So protocols are more than just handshakes and serialization patterns?
Protocols govern all interactions to ensure a system does not get into a inconsistent state. If designed well they can be very flexible, offer high affordance, and allow great things to happen.
We use protocols all the time in our everyday lives. Without protocols we would descend into chaos. Without protocols we would not be able to collectively achieve great things. Even the lowly ant can achieve amazing things as a colony by following very simple protocols.
InfoQ: When we write code, there are protocols all around us, TCP, FTP, HTTP, etc. Typically we deliver our solutions several layers above these protocols, but are there benefits to understanding how they work? Or is a "black box" approach good enough?
It helps to understand the layering of technologies on which our applications rely. We do not need to know each layer in intimate detail, unless you work in layer, or are just passionate and want to understand and learn from others. However we can benefit greatly from understanding what each layer offers and each layers major functional characteristics so our higher level software can work in harmony with the deeper layer rather than against them. I like to call this Mechanical Sympathy.
One way of understanding our communication layers is the OSI (Open Systems Interconnection) layer model. Our applications are typically at layer 7. Often we need to implement application layer protocols to ensure application consistency. Such an example would be adding a sequence number to all transactions between peered applications to allow for the detection of loss or to work in a disconnected fashion. The FIX (Financial Instruments Exchange) protocol has such a message sequencing concept. The people who designed this protocol did not respect, or possibly were not aware of, the OSI layer model. As a result they conflated the session level protocol with the application level protocol which resulted in it being much more difficult to do recovery or support resilient clusters. This is an example of how a lack of understanding of the communications stack can result in systems that are more complex, less robust, and less performant than they could otherwise have been.
InfoQ: When should we consider writing our own protocols?
We are designing protocols all the time. We should consider them more so when we are considering interaction patterns. They go hand in hand with state machines. It is amazing how much an API can be improved when people pause and map out the possible interaction patterns and then document these into a protocol. Calling a method on an object is just passing it a message, albeit in a very coupled way given most programming languages. As we take a more decoupled approach the messages become first class concepts that really benefit from being organised into a protocol. Documenting our protocols help refine them and make a system easier to understand.
InfoQ: Is SBE is a tool to generate efficient and common protocols?
SBE is a codec that affords the very efficient encoding and decoding of messages which make up our protocols. Often people do not decouple a system sufficiently because message encoding can be expensive given the implementation of other codecs. SBE takes the cost of encoding or decoding a message down to a similar cost to accessing the fields of an object in C++ or Java. Additionaly, SBE is allocation free so has no impact on our garbage collectors. As someone who profiles many real-world systems, I've come to realise that encoding/decoding of messages, and resultant garbage, is one of the top 3 performance bottlenecks we all suffer from.
Protocol design and implementation are not only for system architects and low-level platform developers. Introducing thoughtful protocols to a system at its interaction points can have many benefits, one of which being the stablization and a reduction in exception cases. Whether used implicitly within a lower layer, such as TCP, or through an explicit set of instructions that are published across development teams, protocols demonstrate the intent of the system's author and allow participants in a transaction to faithfully execute that intent.