At QCon London 2024, Bernd Ruecker recommended implementing long-running tasks asynchronously with a process-orchestration platform. Such a platform provides better service boundaries and efficiencies and reduces accidental system complexity and risk. Organizing the platform centrally in an organization eases orchestration adoption by applications.
Waiting is the main reason for long-running tasks, and most workflows include it. There are business reasons for this, such as getting customer feedback, and technical ones, such as receiving responses from other systems. The talk used a customer onboarding flow as an example, where one manual step was to approve the customer application. As the presenter noted, "Waiting is a pain." Not only does it slow things down, but applications must also handle timeouts while waiting.
Ruecker then used ordering pizza to illustrate the advantages of asynchronous communication. Ordering by phone is a synchronous, blocking step that temporarily couples both participants. On the plus side, it provides direct feedback. On the other hand, ordering by email is asynchronous and non-blocking, without temporal coupling. However, the downside is that it lacks direct feedback. Either way, the feedback loop is not the result—making and delivering the pizza is.
The speaker argued that an asynchronous system provides a better user experience and allows for better scalability through decoupling. He used another developer's favorite food for illustration: Decoupling speeds up coffee delivery at Starbucks, as one person takes orders and another fulfills them. It also allows scale, as multiple baristas can work on orders simultaneously.
The talk discussed two challenges caused by failures in workflows. Ruecker cited his inability to check into a flight as an example. There, the airline system passed the failure to the top of the call chain – the user. So he tried (and failed) multiple times to repeat the check-in. He speculated that a component error, such as an unavailable barcode generator, caused the failure. In this case, the airline system should have checked him in and tried to generate the ticket with the barcode in the background. Another common challenge is rolling back changes across multiple components, as no global transaction usually exists. For example, if a payment consists of deducting funds from an account and then paying the rest with a credit card, the fund deduction must be undone if the credit card payment fails.
The speaker advocated using BPMN, an ISO-standardized, graphical workflow description language. Graphical workflows document requirements in a way that's understood by both business and technical staff. Dashboards can also use BPMN to visualize the system status and draw messages on the flow chart.
The talk proposed using team interaction modes from the "Team Topologies" book: The Platform team runs and provides the process-orchestration platform as a service, while the enabling team acts as internal consultants who help use the service. The stream-aligned team uses the process-orchestration platform service to provide business value. In some cases, a Complicated Subsystem team has to do the integration work that nobody else wants to do.
Access recorded QCon London talks with a Video-Only Pass.