John Gossman, lead architect at Microsoft Azure, spoke on Thursday at ServerlessConf Conference about Microsoft's open source Serverless platform. The Microsoft team also gave demos on Azure Functions, Logic Apps and how to monitor the platform components.
Spotify clients generate up to 1.5 million events per second at peak hours and all are handled by their Event Delivery System, designed to have a predictable latency and to never lose an event, Igor Maravic noted in his presentation at the recent QCon London conference, where he gave a high level overview of the system and some of the key operational aspects.
The world is concurrent with everything around us asynchronous and event oriented. Concurrency and the cloud are things every developer will have to deal with in the future, Joe Duffy claimed in his keynote at the recent QCon London conference. At the heart of this is communication, which is essential both for concurrent and distributed systems.
Hazelcast, previously known for the open-source caching and in-memory data grid technologies, has announced a major release of their new stream processing engine, Jet.
The Uber Engineering team released their Kafka auditing tool called Chaperone as an open-source project. Chaperone allows for auditing and detection of data loss, latency, and duplication of messages in the multi-datacenter and high-volume Kafka setup at Uber.
Apache Eagle, an open-source solution for identifying security and performance issues on big data platforms, graduates to Apache top level project on January 10, 2017. Firstly open-sourced by eBay on October 2015, Eagle was created to instantly detect access to sensitive data or malicious activities and, to take actions in a timely fashion.
Migrating an existing system towards microservices is very different from building a new micoservices-based system, Joris Kuipers, architect at Trfork Amsterdam, claims in a presentation describing an ongoing process of refactoring a large monolithic application, based on CQRS using Axon framework, towards a microservices architecture.
Domain-Driven Design (DDD) is a great technique bringing designs closer to the domains we are working in, but too often we make early decisions with a focus on structure, which is not the intention of DDD. Instead we should start with the events in a domain, Russ Miles claims when describing the advantages of going “events-first” when building microservices.
Julien Nioche, director of DigitalPebble, PMC member and committer of the Apache Nutch web crawler project, talks about StormCrawler, a collection of reusable components to build distributed web crawlers based on the streaming framework Apache Storm. InfoQ interviewed Nioche, main contributor of the project, to find out more about StormCrawler and how it compares to other similar technologies.
Microsoft recently announced an addition to its Platform as a Service (PaaS) offering called Azure Functions. Initially launched as a preview service in March 2016, Azure Functions provide developers with an event-driven serverless compute platform that allow organizations to pay for only what they consume.
Javier Lopez and Mihail Vieru spoke at Reactive Summit 2016 Conference about cloud-based data integration and distribution platform used for stream processing in business intelligence use cases. Their solution is based on technologies such as Flink, Kafka and Elasticsearch.
Lambda architecture has been a popular solution that combines batch and stream processing. Kartik Paramasivam at LinkedIn wrote about how his team addressed stream processing and Lambda architecture challenges using Apache Samza for data processing. The challenges described are the late arrival of events and the processing of duplicated messages.
Microsoft’s recently open-sourced P language aims to make it possible to write safe asynchronous event-driven programs on Linux, macOS, and Windows.
Reactive microservices, data center scale operating system (DCOS), and staging reactive data pipelines were the highlighted topics at Reactive Summit 2016 Conference held this week. InfoQ team attended the conference and this post is a summary of the first day's events at the conference.
Confluent Enterprise latest version supports multi-datacenter replication, automatic data balancing, and cloud migration capability. Confluent, provider of the Apache Kafka based streaming platform, announced last week the new features for Confluent Enterprise, to help build streaming data pipelines and develop stream processing applications.