BT

Ready for InfoQ 3.0? Try the new design and let us know what you think!

Cloud Follow 377 Followers

Amazon Announces Managed Streaming for Kafka in Public Preview

by Steef-Jan Wiggers Follow 10 Followers on  Dec 06, 2018

At the recent AWS re:Invent 2018 event, Amazon announced a new fully managed service that makes it easy for customers to build and run applications that use Apache Kafka to process streaming data. This new service is called Amazon Managed Streaming for Kafka, Amazon MSK for short, and is now in public preview.

Cloud Follow 377 Followers

Google Cloud Announces Transfer Appliance in Beta for Cloud Data Migrations in the EU

by Steef-Jan Wiggers Follow 10 Followers on  Nov 20, 2018

Google announced that Transfer Appliance, a high-capacity server that lets customers move large amounts of data to Google Cloud Platform (GCP) quickly and securely, is available in beta in the European Union (EU). Google will handle the data transfer with Transfer Appliance in GCP in the EU, and data will not leave the EU.

AI, ML & Data Engineering Follow 1109 Followers

The Evolution of Uber’s 100+ Petabyte Big Data Platform

by Hrishikesh Barua Follow 18 Followers on  Nov 10, 2018

Uber’s engineering team wrote about how their big data platform evolved from traditional ETL jobs with relational databases to one based on Hadoop and Spark. A scalable ingestion model, standard transfer format and a custom library for incremental updates are the key components of the platform.

AI, ML & Data Engineering Follow 1109 Followers

Data Lakes and Modern Data Architecture in Clinical Research and Healthcare

by Srini Penchikala Follow 41 Followers on  Nov 08, 2018

Dr. Prakriteswar Santikary, chief data officer at ERT, spoke at Data Architecture Summit 2018 Conference last month about data lake architecture his team developed at their clinical research organization. He discussed the data platform deployed in the cloud to streamline data collection, aggregation and clinical reporting and analytics, using concepts like serverless computing and data services.

AI, ML & Data Engineering Follow 1109 Followers

Event Sourcing to the Cloud at HomeAway

by Srini Penchikala Follow 41 Followers on  Nov 05, 2018 3

Adam Haines, Data Architect at HomeAway, recently spoke at the Data Architecture Summit 2018 Conference about how his team leverages event sourcing cloud design pattern to accelerate the big data initiatives in their organization.

Cloud Follow 377 Followers

Cloudera and Hortonworks Merge with Goal to Increase Competition with Cloud Offerings

by Alex Giamas Follow 10 Followers on  Oct 31, 2018

Earlier this month, Cloudera and Hortonworks announced an all-stock merger at a combined value of around $5.2 billion. Analysts have argued that this merger is aimed at increased competition that both companies are facing from cloud vendors like Amazon, Google and Microsoft. In this article we log reactions from analysts and the industry, and the implications for current customers.

AI, ML & Data Engineering Follow 1109 Followers

Agile Data Modeling for NoSQL Databases

by Srini Penchikala Follow 41 Followers on  Oct 30, 2018

Pascal Desmarets recently spoke at Data Architecture Summit 2018 Conference about agile modeling and best practices for NoSQL databases.

Cloud Follow 377 Followers

Redis 5.0 Released with New "Streams" Data Type

by Alex Giamas Follow 10 Followers on  Oct 26, 2018

Redis recently announced version 5 of its popular database, 15 months after the release of Redis 4. Probably the most important feature of this version is the support for a new data type, Streams. Sorted set functionality has also improved and Redis modules have also been expanded, with the introduction of Clusters and Timers APIs. LOLWUT and other improvements are reviewed in the article...

JavaScript Follow 478 Followers

Tim Berners-Lee Introduces "Solid" Decentralized Identity Platform

by Dylan Schiemann Follow 13 Followers on  Oct 16, 2018

Solid is a new decentralized identity platform from WWW Creator Tim Berners-Lee. Solid provides a mechanism for users to own and better control the usage of their data.

AI, ML & Data Engineering Follow 1109 Followers

William McKnight on Data Platforms and Creating a Modern Data Architecture

by Srini Penchikala Follow 41 Followers on  Oct 15, 2018

William McKnight gave a keynote presentation last week at Data Architecture Summit 2018 Conference on creating a modern data architecture using different data platforms.

DevOps Follow 1054 Followers

High Volume Space Exploration Time-Series Data Storage in PostgreSQL

by Hrishikesh Barua Follow 18 Followers on  Oct 13, 2018 1

The European Space Agency Science Data Center (ESDC) switched to PostgreSQL with the TimescaleDB extension for their data storage. ESDC’s diverse data includes structured, unstructured and time series metrics running to hundred of terabytes, and querying requirements across datasets with open source tools.

Cloud Follow 377 Followers

Netflix Keystone Real-Time Stream Processing Platform

by Alex Giamas Follow 10 Followers on  Sep 30, 2018

Netflix recently published a post in their tech blog discussing the design considerations and insights of Keystone, their Real-time stream processing platform. Keystone has been operational since December 2015 and has grown significantly over the years as Netflix subscribers have grown from 65 to over 130 million in the past 3 years. This article follows on the latest state of Keystone platform...

Emerging Technologies Follow 100 Followers

Implementing Privacy by Design in Hyperledger Indy

by Kent Weare Follow 11 Followers on  Sep 30, 2018

Centralized identity providers, such as social media sites and consumer email services, provide convenience to users. But this approach creates data privacy and security risks. Hyperledger Indy, an open source blockchain project, is being built to address the current issues that exist in centralized identity providers by taking a 'Privacy by Design' approach to deal with these risks.

AI, ML & Data Engineering Follow 1109 Followers

California Creates Consumer Privacy Act

by Michael Stiefel Follow 6 Followers on  Aug 31, 2018

California has enacted the California Consumer Privacy Act (CCPA) of 2018 which, starting on January 1, 2020, would grant consumers several rights with respect to information about them that businesses collect, store, sold, and share. This is the first legislation of its kind in the United States.

AI, ML & Data Engineering Follow 1109 Followers

New York Creates Task Force to Examine Automated Decision Making

by Michael Stiefel Follow 6 Followers on  Jul 31, 2018

New York City has created an Automated Decision Systems Task Force to demand accountability and transparency in how algorithms are used in city government. The final report of the task force is due in December 2019. This task force is the first in the United States to study this issue.

BT