InfoQ Homepage AI, ML & Data Engineering Content on InfoQ
-
/articles/oracle-to-mysql-aws-database-migration-service/en/smallimage/logo-1662492103321.jpg)
How to Migrate an Oracle Database to MySQL Using AWS Database Migration Service
Data migration efforts are typically taken up for database consolidation, cost considerations, or migrating on-prem databases to a cloud platform. In this article, author Deepak Vohra discusses the details of migrating a local database to MySQL database on the cloud, using AWS Database Migration Service.
-
/articles/auto-ml-promise-reality/en/smallimage/logo-1661333792384.jpg)
AutoML: the Promise vs. Reality According to Practitioners
Automation to improve machine learning projects comes from a noble goal, but true end-to-end automation is not available yet. As a collection of tools, AutoML capabilities have proven value but need to be vetted more thoroughly. Findings from a qualitative study of AutoML users suggest the future of automation for ML and AI rests in the ability for us to realize the potential of AutoMLOps.
-
/articles/business-system-integration/en/smallimage/business-systems-integration-is-about-to-get-a-whole-lot-easier-small-1660901081551.jpg)
Business Systems Integration is about to Get a Whole Lot Easier
A new breed of integration software is arising that syncs business data into a simplified data hub and then syncs that data to the destination system. The benefit of this integration pattern is that it reduces the number of manual transformations required (often to zero) and makes it easier to write manual transformations when you have to.
-
/articles/streaming-first-real-time-ml/en/smallimage/streaming-ml-small-logo-1660833205193.jpeg)
Streaming-First Infrastructure for Real-Time Machine Learning
This article covers the benefits of streaming-first infrastructure for two scenarios of real-time ML: online prediction, where a model can receive a request and make predictions as soon as the request arrives, and continual learning, when machine learning models are capable of continually adapting to change in data distributions in production.
-
/articles/secure-distributed-database-cluster/en/smallimage/logo-1660581268936.jpg)
Creating a Secure Distributed Database Cluster Leveraging Your Existing Database Management System
The emergence of Big Data and data lakes doesn't necessarily mean the disappearance of the trusted relational database. The two can coexist, relational databases just need to adjust. For the transition we propose Database Plus, a new technology & concept applicable to any database, that answers these challenges and eliminates switching costs and vendor lock-in.
-
/articles/change-data-capture-debezium/en/smallimage/logo-1660294927422.jpg)
Debezium and Quarkus: Change Data Capture Patterns to Avoid Dual-Writes Problems
It’s common in microservices to write data in two places, a database and then send the content to another microservice. One approach to tackle this problem is dual writes, but you may lose data because of concurrent writes. Debezium is an open-source project for change data capture using the log scanner approach to avoid dual writes and communicate persisted data correctly between services.
-
/articles/ai-ml-data-engineering-trends-2022/en/smallimage/InfoQ-Trends-Report-1659439712027.jpeg)
AI, ML, and Data Engineering InfoQ Trends Report—August 2022
In this annual report, the InfoQ editors discuss the current state of AI, ML, and data engineering and what emerging trends you as a software engineer, architect, or data scientist should watch. We curate our discussions into a technology adoption curve with supporting commentary to help you understand how things are evolving.
-
/articles/building-neural-networks-tensorflow-net/en/smallimage/logo-1657201887957.jpg)
Building Neural Networks with TensorFlow.NET
TensorFlow is an open-source framework developed by Google scientists and engineers for numerical computing. TensorFlow.NET is a library that provides a .NET Standard binding for TensorFlow. In this article, the author explains how to use Tensorflow.NET to build a neural network.
-
/articles/steampipe-api-sql/en/smallimage/logo-1656952403557.jpg)
API Friction Complicates Hunting for Cloud Vulnerabilities. SQL Makes it Simple
APIs can tell you everything about your cloud infrastructure, but they're hard to use and work in different ways. What if you could write simple SQL queries that call APIs for you and put results into a database? Steampipe, an open-source project that maps APIs to Postgres foreign tables, makes that dream come true. It's hard enough to reason over data. Acquiring it should be easy, and now it is.
-
/articles/cloud-native-apache-dolphinscheduler/en/smallimage/logo-1655898721435.jpg)
Embracing Cloud-Native for Apache DolphinScheduler with Kubernetes: a Case Study
This article shares how Apache DolphinScheduler was updated to use a more modern, cloud-native architecture. This includes moving to Kubernetes and integrating with Argo CD and Prometheus. This improves substantially the user experience of deploying, operating, and monitoring DolphinScheduler.
-
/articles/mlops-deploying-ml-production/en/smallimage/logo-1654663703953.jpg)
What You Should Know before Deploying ML in Production
What should you know before deploying machine learning projects to production? There are four aspects of Machine Learning Operations, or MLOps, that everyone should be aware of first. These can help data scientists and engineers overcome limitations in the machine learning lifecycle and actually see them as opportunities.
-
/articles/ai-for-software-developers/en/smallimage/logo-1652883200735.jpg)
AI for Software Developers: a Future or a New Reality?
In this article, author Nikita Povarov discusses the role AI/ML plays in software development and how tasks like code completion, code search, and bug detection can be powered by machine learning. But he also explains why a complete replacement of programmers by algorithms isn't going happen any time soon.