Blockchain – The Next Big Thing for Middleware
Fascinating new technologies are emerging these days. Everybody talks about cloud, containers, big data and machine learning. Another disrupting technology is blockchain. You might have heard about blockchain as the underlying infrastructure of Bitcoin. But Bitcoin is just the tip of the iceberg. This article explains the use cases and technical concepts behind blockchain, gives an overview about available services, and points out why middleware is a key success factor in this space.
Motivation and Use Cases for Blockchain
More than 700 cryptocurrencies were available for trade in online markets as of 11 July 2016, but only 9 of them had market capitalizations over $10 million. A new term emerged therefore: Altcoin (meaning “Bitcoin Alternatives”). However, Altcoin is not the same as blockchain, but “just” another alternative for financial transactions.
Blockchain is a layer under the hood of a cryptocurrency. It is all around “the transfer of assets within a business network”. Blockchain enables the creation of a secure, trusted, peer-to-peer network between partners or public users to build any kind of distributed business application.
Therefore, Blockchain goes way beyond financial transactions. Here are some examples:
- Lazooz: Ride-sharing for collaborative transportation
- OpenBazaar: Open your own shop without Ebay or Amazon as intermediary
- Skuchain: Next Generation Supply Chain Management (SCM)
- Internet of Things and Smart Contracts: Probably the next “big thing” for blockchain after financial industry
So, why should we care about blockchain instead of keeping the current status in our existing software and applications? We already have Uber, Ebay, and several vendors for SCM and IoT, right? Here are some points to think about why blockchain might be the game changer:
- Innovate and provide new services and distributed applications
- Enable smart contracts to cut out the expensive and / or slow middleman in many industry scenarios streamlining processes, like settlement
- Today we are just scratching the surface of possible use cases. We cannot really imagine yet what customers and partners will build. This is similar to Open API initiatives where we create new added value or even new business models by combining various internal or external APIs
Let’s take a look at an example where this disruption can change a business significantly: Airline transportation and the compensation of customers for delayed or cancelled flights. In many countries, customers have a right for a specific amount of compensation if a flight is delayed. In Europe, you can claim compensation up to €600 per passenger for flight delays of more than three hours. However, the airlines do not (have to) pay this automatically. You have to claim it; often with the support of a lawyer. Therefore, many customers do not claim this money.
Welcome to the world of blockchain where smart contracts process such a scenario automatically and in a secure way. Governments in conjunction with global non-profit airline associations like International Air Transport Association (IATA), which “support aviation with global standards for airline safety, security, efficiency and sustainability,” could enforce airlines to compensate customers automatically as it is defined by law. Of course, the airlines would not be happy in this case, but it is a great example of how blockchain can help to disrupt “business models” respectively enforce specific behavior or business processes.
Technical Concepts and Frameworks
Let’s take a closer look at the technical concepts and frameworks behind blockchain technology. A blockchain is a protocol and ledger for building an immutable historical record of transactions. A blockchain implementation means:
- Ledgers record transactions – the passing of value from owner to owner
- Transactions are time based
- Once a Transaction is recorded you cannot alter it
- You need to be able to detect if your ledger has been altered
- A data structure called the blockchain which serves as the distributed back-end database
- A cryptographic token, e.g. the Bitcoin (BTC) in the Bitcoin protocol or ether (ETH) for Ethereum
- A peer-to-peer network for discovery and communications
- A consensus formation algorithm. Note that many enterprise blockchains will not use mining (like Bitcoin) for consensus
- A virtual machine that enables programmable money in Bitcoin and decentralized applications in Ethereum
IBM Bluemix documentation states, “Blockchain consists of a network, over which members track and exchange assets, and a record of all exchanges (ledger), which is replicated to all participating members. Applications deployed to a blockchain consist of a self-executing contract and a client-side application that interfaces with the network through an SDK or API.” (see figure 1)
Figure 1: A Blockchain Network (Source: IBM)
Blockchain Technologies, Frameworks and Standards
A variety of blockchain technologies exists. They differ regarding:
- Public vs private (internal / partner)
- Adopted vs. new vs. dead
- Different characteristics (speed, security, consensus algorithms, etc.)
There is no blockchain standard yet, but at least there exists some consortiums to build standards, already: R3 Consortium is a blockchain technology company, which leads a consortium of 45 financial companies in research and development of blockchain usage in the financial system. Smart Contracts Alliance is an industry initiative of the Chamber of Digital Commerce, developed to be an authoritative resource for smart contracts.
Two open blockchain frameworks seem to get a lot of traction these days:
- Hyperledger: A collaborative effort created to advance blockchain technology by identifying and addressing important features for a cross-industry open standard for distributed ledgers that can transform the way business transactions are conducted globally.
- Ethereum: A decentralized platform that runs smart contracts: applications that run exactly as programmed without any possibility of downtime, censorship, fraud or third party interference. ConsenSys is the main company behind Ethereum.
Both frameworks have different ideas, analogous to the differences between IaaS and PaaS technologies. Hyperledger is more an Infrastructure as a Service, like Amazon Web Services (AWS) in the public cloud or OpenStack on premise. Ethereum is more a Platform as a Service, like CloudFoundry or OpenShift.
Blockchain as a Service (BaaS)
Most enterprises will not build their own infrastructure for blockchain projects. They will leverage cloud services, which are called Blockchain-as-a-Service (BaaS). IBM and Microsoft are taking a leading role here. They offer cloud platforms for development and enterprise scale based on open technologies and frameworks. Their portfolio includes blockchain offerings plus integration into their other cloud services.
IBM has Blockchain on Bluemix, which leverages the Hyperledger project under the hood. Microsoft Blockchain as a Service on Azure uses the Ethereum platform. In addition, Microsoft partners with various other blockchain vendors, including Ripple, Eris, Coinprism and Factom.
Surprisingly, the other two major cloud vendors, Amazon with Amazon Web Services (AWS) and Google with Google Cloud Platform (GCP), are not very visible in the blockchain market yet. Google invested in the startup Ripple, a blockchain technology for the bank industry. Amazon offers no proprietary blockchain infrastructure service, but only Eris via its marketplace. Eris is a Docker-based, blockchain-backed application platform supporting different blockchain frameworks.
Blockchain and Middleware
Is blockchain just a hype? Will everybody use it, like a web browser or mobile app today? And why is middleware relevant for it? The answer is simple: “Blockchain will be a workhorse, but behind the scenes.” Blockchain will not be relevant from the perspective of most business users, but a key component for distributed infrastructure (and middleware) applications under the hood! Think about how blockchain can be leveraged in an enterprise architecture and you will quickly find out how this related to middleware. There are two key challenges with blockchain:
A blockchain infrastructure is its own independent peer-to-peer network without a central backbone. To integrate with this network, you have to be part of it, i.e. you have to run a blockchain client on your system or integrate with one. Integration is not trivial due to security and governance requirements, the concept of distributed, trusted peers, and a public key infrastructure behind blockchain networks
A blockchain creates new events all the time with technical and business information, but has no central database. To leverage blockchain events, you need to be able to analyze, correlate and act on these events. Beyond integration with the blockchain, you also need to access and visualize these events, plus correlate the information with data from other non-blockchain systems
Because of these challenges, it helps to leverage key middleware features in a real world blockchain project:
- Integration of many different data sources in real time, e.g. blockchain clients, various backends (like ERP, CRM, legacy systems), cloud services, partner APIs, and others
- Gateway and API management
- Combination of different blockchains
- Data discovery to find insights and patterns in historical data, sometimes including machine learning
- Correlation of blockchain and non-blockchain events in real time, sometimes including applying machine learning and analytic models
- Identity and crypto services to ensure secure and governed communication of all stakeholders in the enterprise architecture
Let’s come back to the airline compensation example from above. The airline association would implement the blockchain infrastructure and define smart contracts to enforce airlines to compensate customers when flights are delayed or canceled. The airlines would face many challenges integrating the blockchain infrastructure with their own systems and processes due to all the blockchain-specific concepts and requirements.
Middleware can be leveraged to reduce and solve these challenges as it is build for connecting different systems with various technologies, standards and communication protocols. In addition, middleware can add augmented intelligence via event correlation and visualization to ensure governance requirements or create added value by finding new insights (such as wrong processes, performance issues, malicious usage or additional opportunities).
Reference Architecture for Blockchain and Middleware
Figure 2 shows a reference architecture for blockchain and middleware:
Figure 2: Reference Architecture for Blockchain and Middleware
The reference architecture includes:
- Blockchain networks leveraging different frameworks under the hood like Hyperledger or Ethereum infrastructures
- Cloud Native Integration Layer including API Management to integrate blockchains and others system or cloud services. See this survey of modern application integration architectures to understand current alternative approaches for integration middleware
- Data Discovery / Visual Analytics plus a big data storage (like Apache Hadoop) and machine learning frameworks (like R, Apache Spark ML or H2O.ai) to find insights and patterns in historical blockchain and non-blockchain data. Often, you find the most valuable insights by combining different data sources
- Streaming Analytics to correlate different blockchain and non-blockchain events to apply patterns and analytic models in real time
Various middleware offerings are available on the market:
- Project Bletchley is an open source framework by Microsoft to allow integration of blockchain networks with other applications. It focuses on security and governance requirements and therefore includes features such as a gateway or identity, key and crypto services to allow non-blockchain clients to communicate with the blockchain network.
- Interledger is a lightweight blockchain protocol and integration framework to integrate different blockchains and other interfaces such as banks, Paypal or Skype
- Established middleware vendors like Software AG or TIBCO Software include offerings for integration, event processing / streaming analytics, data discovery / visual analytics, and machine learning, which help integrate and correlate blockchain events with the rest of the enterprise architecture.
A Hackathon Example with Ethereum and Integration Middleware
Let’s take a look at a specific example which we realized at a recent hackathon (see figure 3). The goal was to implement a voting app on a blockchain infrastructure and leverage middleware for integration with non-blockchain clients.
We chose Ethereum as blockchain as it seemed to be the “more complete platform for application development” than Hyperledger right now. However, Hyperledger also has some characteristics, which I really like from a developer perspective, especially that it is based on Docker, offers REST interfaces out-of-the-box, and that you can use Go as the programming language for smart contracts instead of learning a specific smart contract language. In summary, you really need to take a look at the trade-offs of both (and other alternatives on the market) before actually setting up a blockchain infrastructure.
On Ethereum, you develop a so-called Dapp (decentralized app), which consists of two parts: a frontend (written in HTML as web app or any other client using a supported programming language), and a backend (think of it as the ‘database’ for your frontend). A frontend can be a full blockchain peer communicating directly with the backend. The huge benefit here is that every client is a full member of the trusted blockchain network. This is the ideal blockchain world. However, this also makes it much harder to realize and deploy these clients wherever needed - think about Supply Chain Management or Internet of Things scenarios. Therefore, another option is to “hand over the trust” from a full blockchain peer to the middleware and run just a lightweight blockchain client instead which acts as full peer. We combined both options to have full flexibility. In a real world scenario, it might also make sense to offer all functionality to full peers, but only a limited set of operations to everybody via the middleware.
Let’s take a deeper look into our project. First, we installed and configured an Ethereum geth client (as Contract Account) on an AWS EC2 instance to realize the backend of the Dapp. The business logic for the voting app was implemented in Solidity, a specific blockchain language to define smart contracts. Solidity Browser is a nice, simple web IDE, which makes it much easier to get started and realize new smart contracts.
The middleware is also part of the backend node. It integrates with the Ethereum geth client and exposes its interface as REST service to the outside world. This greatly simplies usage, as Ethereum nodes only offer a low-level JSON-RPC interface for communication and allows lightweight non-blockchain peers to communicate with with the backend. We also implemented a web app (deployed via node.js) to display a very basic web user interface to the end user. As it is generally bad practice to maintain an Ethereum node whose RPC protocol listens for connections anywhere else than localhost, it is best to run the middleware on the same instance as the blockchain backend.
Figure 3: Architecture for a Voting App Using Blockchain Peers and Integration Middleware
We realized two frontends. The first one is a full part of the Dapp. This is another Ethereum node (i.e. this is also a full peer of the blockchain network) also realized via a geth client (as Externally Owned Account, EOA) to run on our laptops. These clients can communicate directly with the Dapp’s “backend” nodes where the smart contracts are deployed. While this project only used geth clients (Go client), you could leverage any other Ethereum client, such as cpp-ethereum (C++ client), pyethapp (Python client) or the Ethereum browser Mist which runs DApps just like you interact with websites via a regular browser. Though, you should think about Mist more as a workbench instead of a browser. Note that blockchain is still peer-to-peer, even though the voting app has “frontend” and “backend” clients. Read a more extensive explanation of Ethereum clients to understand the details better.
In addition, in our Hackathon project the laptops of the frontend users were also used for mining to produce new Ether (which is needed to “pay” for blockchain transactions, in this case voting). This is another “bad practice” due to our hardware limitations - and another good reason to use a BaaS service to have explicit mining nodes.
The second web app is no full blockchain peer and communicates with the blockchain backend via the middleware. This is what I expect that most enterprises want to do mostly in real world projects (I do not talk about a global Bitcoin-like project, but more about a blockchain between some partners) because full blockchain peers at the client side have many trade-offs.
While the business logic is very basic, the intention was to show how integration middleware can be leveraged in a blockchain project:
- Expose a REST API via the integration layer (in our case TIBCO BusinessWorks Container Edition, a cloud-native, platform agnostic middleware) to the outside world instead of more complex blockchain APIs. Of course, security requirements and governance need to be matched with the blockchain requirements (e.g. which user can access which blockchain APIs and data).
- Integrate blockchain peers with any other technology, including synchronous Web Services (REST, SOAP), asynchronous messaging (JMS, MQTT), SaaS products (Salesforce, Marketo) or legacy systems (ERP, Mainframe). Note that integration means not just connectivity, but also other core integration requirements such as enterprise-scale, failover, protocol transformation, error handling, and leveraging. design patterns such as Enterprise Integration Patterns or Service Compensation.
- Take over the onus of trust for communication with third-party clients (“hide” the complexity of blockchain security and governance requirements).
- Limit service calls to the blockchain via throttling and caching as every Ethereum transaction costs compute resource (i.e. mining) and money (i.e. Ether, the currency in Ethereum).
Our next steps will be the following to enhance the blockchain experience for end users:
- Add API Management on top of the integration layer to expose APIs via a user-friendly developer portal. The API provider can configure service level agreements and monetization of the available blockchain APIs. This is a pretty simple task if the API Management solution of a software vendor allows out-of-the-box integration with its integration layer.
- Visualize the data to get new insights about blockchain communication and the behaviour of blockchain users and smart contracts. This allows to optimize the business logic (think about fraud detection or other “predictive behaviour”). Correlation via Streaming Analytics might adapt these new insights in real time to new blockchain events.
- Finally, we want to enhance this Hackathon project by setting up a Hyperledger blockchain infrastructure (as a cloud service) and then integrate and correlate data from two different blockchain technologies. While the vision of some blockchain vendors is to have one single, global blockchain, I think the realistic (and better) future is to have different blockchain infrastructures for their relevant use cases like a public payment blockchain (e.g. Bitcoin), but also private blockchains (e.g. smart contracts for Internet of Things or Supply Chain Management between partners).
Blockchain Will Disrupt Enterprises; Middleware is Key for Success
Blockchain is nothing new, but the combination of several existing technologies enables disruption. Bitcoin is just the tip of the iceberg. Blockchain goes way beyond financial transactions and will become relevant in various industries. Internet of things might be the next early adopter by realizing automated smart contracts between partners. Blockchain-as-a-Service offerings allow a quick adoption with fail-fast methodology in proof of concepts and first projects.
A key success factor for blockchains in an enterprise architecture is middleware. You need to integrate blockchains with each other and with many other systems in real time, using different technologies and communication protocols. Data discovery, event correlation, API management, security enforcement and governance are other middleware features to leverage in real world blockchain projects.
If you want to begin your blockchain adventure and take part in one of the most exciting and disrupting technologies these days, I recommend watching the video “Blockchain - The Next Big thing for Middleware,” which explains the content of this article with more graphics and details. Afterwards, get started by spinning up your own blockchain infrastructure via a BaaS cloud service - as a fully-owned setup is complex and error prone. The tooling and IDEs for blockchain technologies are also just maturing these days. Simple web tools like Solidity Browser help a lot in the beginning. Setting up your own blockchain infrastructure and integration with your typical development environment and build pipeline might be a second step, after you have implemented and deployed some first smart contracts.
About the Author
Kai Wähner is Technology Evangelist and Community Director for TIBCO Software - a leading provider of integration and analytics middleware. His main area of expertise lies within the fields of Big Data, Advanced Analytics, Machine Learning, Integration, SOA, Microservices, BPM, Cloud, Internet of Things and Programming Languages such as Java EE, Groovy or Golang. He regularly write about new technologies, articles and conference talks on his blog.