InfoQ Homepage Data Content on InfoQ

Articles

RSS Feed

Newer Older

AI, ML & Data Engineering

DynamoDB Data Transformation Safety: from Manual Toil to Automated and Open Source

Data transformation remains a continuous challenge in engineering and built upon manual toil. The open source utility Dynamo Data Transform was built to simplify and build safety and guardrails into data transformation for DynamoDB based systems––built upon a robust manual framework that was then automated and open sourced. This article discusses the challenges with Data Transformation.

Guy Braunstain
on Nov 23, 2022 1
Architecture & Design

Data Manipulation with Functional Programming and Queries in Ballerina

Ballerina has been designed as a data-oriented programming language and supports a functional programming coding style. The Ballerina query language is similar to SQL in the sense that a query expression is made up of clauses. The Ballerina “Table” data structure can be more effective than maps in representing indexed data collections.

Yehonathan Sharvit
on Aug 11, 2022
Architecture & Design

Ballerina: a Data-Oriented Programming Language

Ballerina’s flexible type system brings the best of statically typed and dynamically typed languages in terms of safety, clarity, and speed of development. Ballerina treats data as a first-class citizen that can be created without extra ceremony, just like strings and numbers.

Yehonathan Sharvit
on May 10, 2022 2
Architecture & Design

Software Architecture and Design InfoQ Trends Report—April 2022

An overview of how the InfoQ editorial team sees the Software Architecture and Design topic evolving in 2022, with a focus on what architects are designing for today.

Thomas Betts Eran Stiller Vasco Veloso Daniel Bryant Sarah Wells
on Apr 04, 2022 4
Cloud

Data Patterns for the Edge: Data Localization, Privacy Laws, and Performance

With growing competition to get data that power experiences to the end-user closer and closer and the advent of local data privacy laws, let's look at different enterprise data patterns like “synchronous data retrieval”, “subsequent data retrieval” and “prefetch data retrieval” on data center.

Anoop Koloth
on Mar 21, 2022
AI, ML & Data Engineering

Developing Deep Learning Systems Using Institutional Incremental Learning

Institutional incremental learning promises to achieve collaborative learning. This form of learning can address data sharing and security issues, without bringing in the complexities of federated learning. This article talks about practical approaches which help in building an object detection system.

Ritesh Sinha
on Jan 05, 2022 2
AI, ML & Data Engineering

AI, ML and Data Engineering InfoQ Trends Report - August 2021

How AI, ML and Data Engineering are evolving in 2021 as seen by the InfoQ editorial team. Topics discussed include deep learning, edge deployment of machine learning algorithms, commercial robot platforms, GPU and CUDA programming, natural language processing and GPT-3, MLOps, and AutoML.

Roland Meertens Kimberly McGuire Srini Penchikala Rags Srinivas Anthony Alford
on Aug 09, 2021
Culture & Methods

GitHub’s Journey from Monolith to Microservices

This article explores GitHub's recent journey towards a microservices architecture. It takes a deeper look at GitHub’s historical and current state, goes over some internal and external factors, and discusses practical consideration points in how Github tackled their migration, including key concepts and best practices of implementing microservices architecture.

Sha Ma
on Jul 28, 2021 1
Culture & Methods

Data-Driven Decision Making – Optimizing the Product Delivery Organization

The Data-Driven Decision Making Series provides an overview of how the three main activities in the software delivery - Product Management, Development and Operations - can be supported by data-driven decision making. Applying Hypotheses, CD Indicators and SRE’s SLIs / SLOs enables a software delivery organization to optimize for effectiveness, efficiency and service reliability.

Vladyslav Ukis
on Apr 20, 2020
Culture & Methods

Data-Driven Decision Making – Product Operations with Site Reliability Engineering

The Data-Driven Decision Making Series provides an overview of how the three main activities in the software delivery - Product Management, Development and Operations - can be supported by data-driven decision making. In Operations, SRE’s SLIs and SLOs can be used to steer the reliability of services in production.

Vladyslav Ukis
on Mar 25, 2020 3
Culture & Methods

Article Series: Data-Driven Decision Making

The Data-Driven Decision Making Series provides an overview of how the three main activities in the software delivery - Product Management, Development and Operations - can be supported by data-driven decision making.

Vladyslav Ukis
on Mar 06, 2020
Culture & Methods

Q&A on the Book Agile Machine Learning

The book Agile Machine Learning by Eric Carter and Matthew Hurst describes how the guiding principles of the Agile Manifesto have been used by machine learning teams in data projects. It explores how to apply agile practices for dealing with the unknowns of data and inferencing systems, using metrics as the customer.

Ben Linders Matthew Hurst
on Feb 12, 2020