Lambda architecture has been a popular solution that combines batch and stream processing. Kartik Paramasivam at LinkedIn wrote about how his team addressed stream processing and Lambda architecture challenges using Apache Samza for data processing. The challenges described are the late arrival of events and the processing of duplicated messages.
Apache Kafka and Kafka Streams frameworks help with developing stream-centric architectures and distributed stream processing applications. Jay Kreps, CEO of Confluent, gave the keynote presentation on stream processing and microservices at Reactive Summit 2016 Conference last week.
Reactive microservices, data center scale operating system (DCOS), and staging reactive data pipelines were the highlighted topics at Reactive Summit 2016 Conference held this week. InfoQ team attended the conference and this post is a summary of the first day's events at the conference.
Confluent Enterprise latest version supports multi-datacenter replication, automatic data balancing, and cloud migration capability. Confluent, provider of the Apache Kafka based streaming platform, announced last week the new features for Confluent Enterprise, to help build streaming data pipelines and develop stream processing applications.
Scalability should be considered when developing a Minimum Viable Product (MVP). An MVP needs to be technically scalable and you need to have a plan on how to scale quickly when your MVP attracts many users and becomes successful. Knowing your possible performance bottlenecks and using common sense while developing your MVP will get you very far, says Erik Duindam, CTO at Unboxd.
Real-time analysis of event streams has a new focus in Big Data platforms, both on-premise and in the cloud. AWS have released Amazon Kinesis Analytics, a rival to Azure StreamAnalytics. Both platforms use a simple SQL language for complex querying, and move Big Data analysis into a SaaS-like space.
A team of scientists at IBM Research in Zurich, have created an artificial version of neurons using phase-change materials to store and process data. These phase change based artificial neurons can be used to detect patterns and discover correlations in Big Data (real-time streams of event based data) and unsupervised machine learning at high speeds using very little energy.
Data science is about the data that you need; deciding which data to collect, create, or keep is fundamental argues Lukas Vermeer, an experienced Data Science professional and Product Owner for Experimentation at Booking.com. True innovation starts with asking big questions, then it becomes apparent which data is needed to find the answers you seek.
You can now run Microsoft SQL Server on Google Cloud Platform. All the major cloud providers support SQL Server, either with an IAAS or PAAS model, but the offerings have very different feature sets. Providing an attractive option for migrating SQL Server to the cloud seems to be a new focus for Google, Amazon and Microsoft.
In the webcast entitled "What's Better Than Microservices? Serverless Microservices," Alan Williams (Autodesk), Asha Chakrabarty (Amazon) and Alan Ho (Apigee) discuss the architecture of a serverless microservice built with lambda functions with Apigee end-points running on AWS.
In a recent blog post, Uber detailed why they have chosen to replace PostgreSQL with MySQL. In this article we present a high level overview of the problems they faced, how MySQL fares against them and some of the best rebuttals against the case presented by Uber.
AWS released an update to its Relational Database Service (RDS) that allows users to take advantage of native SQL Server backup and restore functionality. AWS beat Microsoft Azure to market with this capability which simplifies database migration and disaster recovery scenarios.
On July 15th, Microsoft announced the Azure Premium Messaging service has reached General Availability (GA). Premium Messaging targets customers who would like more predictable messaging performance. InfoQ reached out to Dan Rosanova, Principal Program Manager on the Azure Service Bus team for additional insight into this milestone.
InfoQ's Rags Srinivas talks to Basho's CTO Dave McCrory about the open sourcing of Riak TS 1.3 which is geared to handle time series data.