InfoQ Homepage Data Content on InfoQ
-
Apache Spark Brings Pandas API with Version 3.2
The Apache Spark team has integrated the Pandas API in the product's latest 3.2 release. With this change, dataframe processing can be scaled to multiple clusters or multiple processors in a single machine using the PySpark execution engine.
-
Cloudera Announces the General Availability of Cloudera DataFlow for the Public Cloud
The enterprise data cloud company Cloudera recently announced the general availability (GA) of Cloudera DataFlow for the Public Cloud, a cloud-native service for data flows to process hybrid streaming workloads on the Cloudera Data Platform (CDP).
-
Microsoft Renames Its Azure for FHIR API to Azure Healthcare APIs
Recently Microsoft announced the renaming of its Cloud for Healthcare's Azure API for Fast Healthcare Interoperability Resource (FHIR) to "Azure Healthcare APIs." In addition to the renaming of the APIs, the company also expands support for healthcare data to include patient health data via FHIR, medical imaging data via DICOM - and medical device data via the Azure IoT Connector for FHIR .
-
Perceiver: One Neural-Network Model for Multiple Input Data Types
Google’s DeepMind company has recently released a state-of-the-art deep-learning model called Perceiver that receives and processes multiple input data ranging from audio to images, similarly to how the human brain perceives multimodal data. Perceiver is able to receive and classify input multiple data types, namely point cloud, audio and images.
-
The Journey from Monolith to Microservices at GitHub: QCon Plus Q&A
GitHub needed to fundamentally rethink how they did software development due to all of the different cultures, norms, and technology stacks that their teams brought to the table. They are migrating toward a microservices architecture that enables different teams and systems and technologies to work harmoniously together.
-
Google Announces a New, More Services-Based Architecture Called Runner V2 to Dataflow
Google Cloud Dataflow is a fully-managed service for executing Apache Beam pipelines within the Google Cloud Platform(GCP). In a recent blog post, Google announced a new, more services-based architecture called Runner v2 to Dataflow – which will include multi-language support for all of its language SDKs.
-
The Distributed Data Mesh as a Solution to Centralized Data Monoliths
Instead of building large, centralized data platforms, corporations and data architects should create distributed data meshes.
-
Data Science at the Intersection of Emerging Technologies
Kirk Borne, principal data scientist at Booz Allen Hamilton, gave a keynote presentation at this year’s Oracle Code One Conference on how the connection between emerging technologies, data, and machine learning are transforming data into value. Emerging technological innovations like AI, robotics, computer vision and more, are enabled by data and create value from data.
-
The Future of Data Engineering: Chris Riccomini at QCon San Francisco
At QCon San Francisco 2019, Chris Riccomini presented “The Future of Data Engineering”. The key takeaway of his talk is about reaching an end goal with data engineering, which is having a fully automated decentralized data warehouse.
-
Lessons Learned from Innovating at Google: Frame the Problem, Use Data, and Define the MVP
The truly great, innovative, useful ideas come mostly from two sources: your target users, and people working in the organization - not necessarily those with a "product manager" hat. Experimentation can help us to materialize ideas into actual products and technology. Framing the problem, using data, and defining the MVP can help us to increase the chance of success in innovation.
-
Microsoft Announces Public Preview of Azure Data Share
Microsoft has announced the public preview of Azure Data Share, which provides capabilities to share data with users in the own organization, as well as with other organizations. Essentially, Microsoft positions the recently announced service as a big data tool, though it’s also possible to share individual files.
-
DevOps Needs Continuous Improvement to Succeed
Continuous improvement is not a new thing and is often misunderstood. To be successful, we can take guidance from agile principles and apply them to the DevOps world, argued Mirco Hering, managing director at Accenture. At Agile Portugal 2019 he spoke about DevOps leadership in the age of agile.
-
Los Angeles CTO Roundtable about AI and Data
The recent "Leaders in Data CTO Roundtable" in Los Angeles included discussions about an artificial intelligence (AI) framework/platform for business, data in the next five years, data software stacks, and acquiring data talent.
-
Introducing TensorFlow Privacy, a New Machine Learning Library for Protecting Sensitive Data
In a recent blog post, TensorFlow announced TensorFlow Privacy, an open source library that allows researchers and developers to build machine learning models that have strong privacy. Using this library ensures user data are not remembered through the training process based upon strong mathematical guarantees.
-
Microsoft Announces New Azure Analytics Services ADLS, ADX and More
Microsoft has announced the general availability of two new Azure analytics services - Azure Data Lake Storage Gen2 (ADLS) and Azure Data Explorer (ADX). Furthermore, Microsoft also announced the preview of Azure Data Factory Mapping Data Flow.