Stream Processing and Lambda Architecture Challenges

by Alexandre Rodrigues on  Oct 19, 2016 3

Lambda architecture has been a popular solution that combines batch and stream processing. Kartik Paramasivam at LinkedIn wrote about how his team addressed stream processing and Lambda architecture challenges using Apache Samza for data processing. The challenges described are the late arrival of events and the processing of duplicated messages.

Jay Kreps on Distributed Stream Processing with Apache Kafka and Kafka Streams

by Srini Penchikala on  Oct 16, 2016

Apache Kafka and Kafka Streams frameworks help with developing stream-centric architectures and distributed stream processing applications. Jay Kreps, CEO of Confluent, gave the keynote presentation on stream processing and microservices at Reactive Summit 2016 Conference last week.

Security Insights into the LXD Container Hypervisor

by Hrishikesh Barua on  Oct 15, 2016

A presentation at the Linux security summit last month covered various issues and edge cases with container security in LXD, a container hypervisor from Canonical that uses Linux Containers (LXC) under the hood.

Reactive Summit 2016 Conference: Reactive Microservices and Staging Data Pipelines

by Srini Penchikala on  Oct 08, 2016

Reactive microservices, data center scale operating system (DCOS), and staging reactive data pipelines were the highlighted topics at Reactive Summit 2016 Conference held this week. InfoQ team attended the conference and this post is a summary of the first day's events at the conference.

Confluent Announces Kafka for the Enterprise with Multi-Datacenter Replication

by Srini Penchikala on  Oct 05, 2016

Confluent Enterprise latest version supports multi-datacenter replication, automatic data balancing, and cloud migration capability. Confluent, provider of the Apache Kafka based streaming platform, announced last week the new features for Confluent Enterprise, to help build streaming data pipelines and develop stream processing applications.

Consul 0.7 Adds Atomic K/V Updates, ACL Replication and Improved Protocol Robustness

by Grischa Ekart on  Sep 22, 2016

Last week, Hashicorp released version 0.7 of Consul its open-source distributed service discovery and configuration tool. Tagged a "very large release", it introduces transactions for key/value updates, replication of ACLs across datacenters, improvements to its Raft and Gossip protocol implementations and optimisation of corresponding timings.

Amazon Kinesis Analytics is Like SaaS for Big Data Analysis

by Elton Stoneman on  Sep 14, 2016

Real-time analysis of event streams has a new focus in Big Data platforms, both on-premise and in the cloud. AWS have released Amazon Kinesis Analytics, a rival to Azure StreamAnalytics. Both platforms use a simple SQL language for complex querying, and move Big Data analysis into a SaaS-like space.

IBM Creates Artificial Neurons from Phase Change Memory for Cognitive Computing

by Srini Penchikala on  Sep 13, 2016

A team of scientists at IBM Research in Zurich, have created an artificial version of neurons using phase-change materials to store and process data. These phase change based artificial neurons can be used to detect patterns and discover correlations in Big Data (real-time streams of event based data) and unsupervised machine learning at high speeds using very little energy.

Getting the Data Needed for Data Science

by Ben Linders on  Sep 02, 2016

Data science is about the data that you need; deciding which data to collect, create, or keep is fundamental argues Lukas Vermeer, an experienced Data Science professional and Product Owner for Experimentation at True innovation starts with asking big questions, then it becomes apparent which data is needed to find the answers you seek.

Azure Logic Apps Reaches General Availability

by Kent Weare on  Aug 08, 2016

On July 27th, Microsoft announced their Integration Platform as a Service (iPaaS) offering, Logic Apps has reached General Availability (GA). The GA release includes additional management support, telemetry events, alerts, and consumption-based pricing. InfoQ reached out Jim Harrer, principal group program manager at Microsoft to gain further insight into this Logic Apps release.

Azure Premium Messaging Service Reaches General Availability

by Kent Weare on  Jul 31, 2016

On July 15th, Microsoft announced the Azure Premium Messaging service has reached General Availability (GA). Premium Messaging targets customers who would like more predictable messaging performance. InfoQ reached out to Dan Rosanova, Principal Program Manager on the Azure Service Bus team for additional insight into this milestone.

Basho Open Sources Time Series Database Riak TS 1.3

by Rags Srinivas on  Jul 15, 2016

InfoQ's Rags Srinivas talks to Basho's CTO Dave McCrory about the open sourcing of Riak TS 1.3 which is geared to handle time series data.

Meson Workflow Orchestration and Scheduling Framework for Netflix Recommendations

by Srini Penchikala on  Jul 10, 2016

Netflix's goal is to predict what you want to watch before you watch it. They do this by running a number of machine learning (ML) workflows every day. Meson is a workflow orchestration and scheduling framework that manages the lifecycle of all these machine learning pipelines that build, train and validate personalization algorithms to help with the video recommendations.

Google BigQuery Now Allows to Query All Open-Source Projects on GitHub

by Sergio De Simone on  Jul 08, 2016 2

A full snapshot of more than 2.8 million open source project hosted on GitHub is now available in Google’s BigQuery, Google and GitHub announced. This will make it possible to query almost 2 billion source files hosted on GitHub using SQL.

Docker Swarm Is Dead. Long Live Docker Swarm.

by Elton Stoneman on  Jun 28, 2016

At DockerCon, Docker released version 1.12 of the core product, Docker Engine. The biggest new feature is that Docker Swarm is no longer a separate tool - now it's built into Docker Engine, making it easier to combine multiple Docker hosts into a single logical unit for increased scale and reliability.

General Feedback
Marketing and all content copyright © 2006-2016 C4Media Inc. hosted at Contegix, the best ISP we've ever worked with.
Privacy policy

We notice you're using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.