InfoQ Homepage Distributed Systems Content on InfoQ
-
The Mechanics of Testing Large Data Pipelines
Mathieu Bastian explores the mechanics of unit, integration, data and performance testing for large, complex data workflows, along with the tools for Hadoop, Pig and Spark.
/presentations/patterns-data-workflows/en/mediumimage/Matbig.JPG)
-
Connecting Stream Processors to Databases
Gian Merlino discusses stream processors and a common use case - keeping databases up to date-, the challenges they present, with examples from Kafka, Storm, Samza, Druid, and others.
/presentations/stream-processors-databases/en/mediumimage/Giabig.JPG)
-
A Brief History of Chain Replication
Christopher Meiklejohn talks through a history of chain replication, starting with the original work from 2004 by van Renesse and Schneider up to new and unique designs of chain replication.
/presentations/chain-replication/en/mediumimage/Chrisbig.JPG)
-
The Raft Consensus Algorithm
Diego Ongaro introduces Raft, a consensus algorithm for managing a replicated log by separating the key elements of consensus and reducing the number of states that must be considered.
/presentations/raft-consensus-algorithm/en/mediumimage/Diebig.JPG)
-
Architecting Distributed Databases for Failure
Fangjin Yang covers common problems and failures seen with distributed systems, and discusses design patterns that can be used to maintain data integrity and availability when everything goes wrong.
/presentations/data-integrity-distributed-systems/en/mediumimage/Fanbig.JPG)
-
Hadoop Workflows and Distributed YARN Apps using Spring Technologies
The authors discuss how Spring for Apache Hadoop can make developing workflows with Map Reduce, Spark, Hive and Pig jobs easier, and using Spring Cloud to build distributed apps for YARN.
/presentations/spring-hadoop-workflow-yarn/en/mediumimage/doibig.jpg)
-
Federated Queries with HAWQ - SQL on Hadoop and Beyond
Christian Tzolov shows different integration approaches between HAWQ and GemFire, showing using Spring XD to ingest GemFire data into HDFS and using Spring Boot to implement a RESTful proxy for HAWQ.
/presentations/federated-queries-hawq/en/mediumimage/Chrisbig.JPG)
-
Building Blocks of a Distributed System
Oren Eini discusses the building blocks of a reliable, transactional distributed database, covering ACID compliance, consistency, failure handling, monitoring, management, and more.
/presentations/distributed-database-system/en/mediumimage/Orebig.JPG)
-
Distributed Java Systems in Minutes with Hazelcast
This talk presents Hazelcast, an open-source distributed Java in-memory container that allows multiple processes to share data using standard Java APIs such as Maps, Sets and Lists.
/presentations/hazelcast-distributed-system/en/mediumimage/Davidbig.JPG)
-
So We Hear You Like Papers
Ines Sombre and Caitie McCaffrey offer a guided tour of papers from past and present research that have reshaped the way we think about building large scale distributed systems.
/presentations/papers-large-distributed-systems/en/mediumimage/Doibig.jpg)
-
Functional Distributed Programming with Irmin
Anil Madhavapeddy introduces the Irmin library by means of a functional queue, shows how the Git mirroring works, and then demonstrates some more complex applications.
/presentations/irmin/en/mediumimage/Anilbig.JPG)
-
Scaling Distributed Systems
Natalia Chechina outlines features of actor and functional programming models, and the reason these models attract so much interest in parallel, concurrent, and scaling world.
/presentations/actor-functional-scale/en/mediumimage/Natbig.JPG)