Elephant in the Cloud - Hadoop as a Service
Hadoop and other big data technologies revolutionized the way organizations run data analytics but the organizations are still facing challenges with operating costs of using these technologies for on-premise data processing. Ashish Thusoo recently spoke at Enterprise Data World Conference about Hadoop as a service offering that helps organizations bridge the gaps with these capabilities.
Google Cloud Machine Learning and Tensor Flow Alpha Release
Late last month Google released an alpha version of their TensorFlow (TF) integrated cloud machine learning service as a response to a growing need to make their Tensor Flow library to run at scale on the Google Cloud Platform (GCP). Google describes several new feature sets around making TF usage scale by integrating several pieces of the GCP like Dataproc, a managed Hadoop and Spark service.
Apache Storm Reaches 1.0, Brings Improved Performance, Many New Features
Version 1.0 is "a major milestone in the evolution of Apache Storm", writes Apache Software Foundation VP for Apache Storm P. Taylor Goetz, and it includes many new features and improvements. In particular, Goetz claims a 3x–16x boost in performance.
GitHub’s DGit Improves Reliability, Performance, and Availability
GitHub has been quietly rolling out DGit, short for “distributed Git”, a new distributed storage system built on top of Git with the aim of improving reliability, availability, and performance of using GitHub.
LFE Brings Lisp to the Erlang Virtual Machine
After 8 years of development, Lisp Flavoured Erlang (LFE) has reached version 1.0, bringing stable support for Lisp programming on the Erlang virtual machine (BEAM). LFE was created by Robert Virding, one of the initial developers of Erlang. InfoQ has spoken with Duncan McGreggor, current maintainer of LFE.
Apache Flink 1.0.0 is Released
InfoQ's Rags Srinivas caught up with Stephan Ewen, a project committer for Apache Flink about the 1.0.0 Release and the roadmap
Anti-Patterns Working with Microservices
The main problem with monolithic applications is that they are hard to scale, in terms of the application, but more importantly, in terms of the team. The main reason for a switch to microservices should be about teams, Tammer Saleh claimed at the recent QCon London conference when describing common microservices anti-patterns and solutions he has encountered.
Microservices for a Streaming World
Embrace decentralization, build service-based systems and attack the problems that come with distributed state using stream processing tools, Ben Stopford urged in his presentation at the recent QCon London conference.
Real-World Consistency Explained: Uwe Friedrichsen Discusses His Favourite Academic Papers
At the microXchg 2016 conference, held in Berlin, Germany, Uwe Friedrichsen presented a deep-dive into “real-world consistency explained”. Friedrichsen referenced multiple academic papers and discussed topics such as ACID vs BASE, his belief that many developers may not fully understand consistency guarantees with a typical SQL database, and how consistency affects microservice systems.
Moving from Transactions to Streams to Gain Consistency
With many databases in a system they are rarely independent from each other, instead pieces of the same data are stored in many of them. Using transactions to keep everything in sync is a fragile solution. Working with a stream of changes in the order they are created is a much simpler and more resilient solution, Martin Kleppmann stated in his presentation at the recent QCon London conference.
"Surviving Microservices" with Richard Rodger at microXchg: Messages, Pattern Matching and Failure
At the microXchg 2016 conference, held in Berlin, Germany, Richard Rodger presented “Surviving Microservices”, a practical guide for developers wanting to keep their microservices architectures ‘healthy and performant’. Key topics discussed in the talk included the benefits of message-oriented systems, pattern matching with inter-service communication, dealing with failure, and Seneca.js.
Don’t Think like an Engineer When Designing Microservices
When designing microservices and their APIs, you need to think like a designer focusing on the users, Nic Benders claimed in his presentation at the recent Microservices Practitioner Summit. Design the API first, then build your services with an outside-in approach.
Microservices and Teams at Amazon
The microservices pattern are changing how we build applications and team structure is extremely important to be successful in building and running these microservices, Chris Munns stated in a talk about how microservices at enterprise scale are built at Amazon at the earlier I Love APIs 2015 conference.
IronFleet: A Methodology for Proving Distributed Systems
A group of researchers from Microsoft has published the paper “IronFleet: Proving Practical Distributed Systems Correct” (PDF) and made available the accompanying source code demonstrating the use of the methodology in machine proving the correctness of a non-trivial distributed system from a safety and liveliness point of view.
Yahoo! Benchmarks Apache Flink, Spark and Storm
Yahoo! has benchmarked three of the main stream processing frameworks: Apache Flink, Spark and Storm.
