With many databases in a system they are rarely independent from each other, instead pieces of the same data are stored in many of them. Using transactions to keep everything in sync is a fragile solution. Working with a stream of changes in the order they are created is a much simpler and more resilient solution, Martin Kleppmann stated in his presentation at the recent QCon London conference.
At the microXchg 2016 conference, held in Berlin, Germany, Richard Rodger presented “Surviving Microservices”, a practical guide for developers wanting to keep their microservices architectures ‘healthy and performant’. Key topics discussed in the talk included the benefits of message-oriented systems, pattern matching with inter-service communication, dealing with failure, and Seneca.js.
When designing microservices and their APIs, you need to think like a designer focusing on the users, Nic Benders claimed in his presentation at the recent Microservices Practitioner Summit. Design the API first, then build your services with an outside-in approach.
The microservices pattern are changing how we build applications and team structure is extremely important to be successful in building and running these microservices, Chris Munns stated in a talk about how microservices at enterprise scale are built at Amazon at the earlier I Love APIs 2015 conference.
A group of researchers from Microsoft has published the paper “IronFleet: Proving Practical Distributed Systems Correct” (PDF) and made available the accompanying source code demonstrating the use of the methodology in machine proving the correctness of a non-trivial distributed system from a safety and liveliness point of view.
Yahoo! has benchmarked three of the main stream processing frameworks: Apache Flink, Spark and Storm.
After several years of development, MBrace 1.0 was released last week. MBrace is a programming model for scalable cloud data scripting and programming with F# and C#. The project consists mainly of code libraries and cloud providers runtime.
The second Microservices Conference arranged by Skills Matter is due early November with two days in Stockholm and London respectively. The list of speakers include the program lead for the conference Russ Miles, David A. Dawson, Björn Carlson, chief architect at Klarna, Viktor Klang, chief software architect at Typesafe, Ian Cooper and Daniel Bryant.
When working with Hadoop, with or without Hunk, there are a number of ways you can accidentally kill performance. While some of the fixes require more hardware, sometimes the problems can be solved simply by changing the way you name your files.
Splunk can now store archived indexes on Hadoop. At the cost of performance, this offers a 75% reduction in storage costs without losing the ability to search the data. And with the new adapters, Hadoop tools such as Hive and Pig can process the Splunk-formatted data.
Splunk opened their big data conference with an emphasis on “making machine data accessible, usable, and valuable to everyone”. This is a shift from their original focus: indexing arbitrary big data sources. Reasonably happy with their ability to process data, they want to ensure that developers, IT staff, and normal people have a way to actually use all of the data their company is collecting.
Preparing for problems like partial failure is the best thing you can do when working with distributed systems, Vaughn Vernon explains in a conversation with InfoQ and refers to a blog post by Jeff Hodges noting its down-to-earth approach and practical advices e.g. designing for partial availability, and using capped exponential back off to restore full operation when dependencies are unavailable.
GameAnalytics, maker of a free analytics platform, has recently open sourced gascheduler an Erlang library that provides a generic scheduler for parallel execution of distributed tasks. InfoQ has spoken to Chris de Vries, one of gascheduler’s creators.
Looking at Command Query Responsibility Segregation (CQRS) in a larger architectural context there are other architectural styles available. There are database technologies solving the same problems but in a simpler way, Udi Dahan states looking into ways of approaching CQRS. There is also a way that fulfils a lot of the CQRS goals but with fewer moving parts when CQRS is really needed.
ELIoT (Extensible Language for the Internet of Things) is a simple and small programming language aiming to make distributed programming easier. A program in ELIoT may appear as a sigle program, but it actually runs on different computers, so, e.g., a variable or function declared on one computer is transparently used on another.