InfoQ Homepage AI, ML & Data Engineering Content on InfoQ
-
Expo: Real Time A/B Testing and Monitoring with Spark Streaming and Kafka at Walmart Labs
The WalmartLabs engineering team developed a real time A/B testing tool called Expo that collects and analyzes user engagement metrics. It uses Spark Structured Streaming to process the incoming data and stores the metrics in KairosDB.
-
Databricks MLflow Integration Now Generally Available
Databricks recently made MLflow integration with Databrick notebooks generally available for its data engineering and higher subscription tiers. The integration combines the features of MLflow with those of Databrick notebooks and jobs. MLflow provides the following three main capabilities: experiment tracking, projects, and MLflow models.
-
Microsoft Launches Several New Machine Learning Services and Extends Its Cognitive Services
Before its Build Developer Conference, Microsoft released several new Machine Learning services and Cognitive Services updates, ranging from no-code tools to hosted notebooks, with several new APIs and other services in-between.
-
OpenAI Introduces Sparse Transformers for Deep Learning of Longer Sequences
OpenAI has developed the Sparse Transformer, a deep neural-network architecture for learning sequences of data, including text, sound, and images. The networks can achieve state-of-the-art performance on several deep-learning tasks with faster training times.
-
Making Robots More Intelligent, Microsoft Releases Autonomous Systems Platform
At the recent Build conference in Seattle, Microsoft announced, in limited preview, an end-to-end toolchain to help developers and organizations build autonomous systems for their industries. The platform includes machine teaching tools and simulation technologies that enable intelligent robotic systems to complete tasks like running autonomous forklifts and robotic inspection platforms.
-
ML.NET, an Open Source Machine Learning Framework for the .NET Ecosystem: Pranav Rastogi Q&A
Earlier this month Microsoft released the first major version of ML.NET, an open source machine learning (ML) framework for the .NET ecosystem. ML.NET allows the development of custom ML models using either C# or F#. These models can be used in scenarios involving sentiment analysis, fraud and spam detection, product and movie recommendation, image classification, and more.
-
Databricks Open Sources Delta Lake to Make Data Lakes More Reliable
Databricks recently announced open sourcing Delta Lake, their proprietary storage layer, to bring ACID transactions to Apache Spark and big data workloads. Databricks is the company behind the creators of Apache Spark, while Delta Lake is already being used in several companies like McAffee, Upwork etc . Delta Lake is addressing the heterogeneous data problem that data lakes often have...
-
Microsoft Open-Sources Approximate Nearest Neighbor Search Algorithm Powering Bing
Microsoft's latest contribution to open source, Space Partition Tree And Graph (SPTAG), is an implementation of the approximate nearest neighbor search (NNS) algorithm that is used in Microsoft Bing search engine.
-
QCon San Francisco 2019: Registrations Open & Top Videos from QCon SF 2018
QCon San Francisco, the software conference that attracts attendees from all over the world, returns November 11-13, 2019, for the 13th year. QCon is organized by the people behind InfoQ & is dedicated to providing a platform for innovators & early adopters to share their story in the global epicenters of software development: Beijing, London, New York, Sao Paulo, Shanghai, and San Francisco.
-
Xipeng Shen on a New Technique to Reduce Deep-Learning Training Time
Researchers at North Carolina State University recently presented a paper at the 35th IEEE International Conference on Data Engineering (ICDE 2019) on their new technique that can reduce training time for deep-neural-networks by up to 69%.
-
Micronaut 1.1 Features Enhanced Support for Building Cloud-Native Applications
During the recent Google Cloud Next conference, Object Computing, Inc. (OCI) announced the release of Micronaut 1.1 featuring support for gRPC, GraphQL, Google Cloud Platform (GCP), RabbitMQ and Amazon Web Services (AWS). There is also a new Bean Introspection API that replaces the JDK Introspector class and new templates for the Micronaut Test project.
-
Investigating Near Misses to Prevent Disasters: QCon London Q&A
Investigating near misses by gathering data from the field and exploring anything that looks wrong or is a bit odd can help to prevent disasters, said Ed Holland, software development manager at Metaswitch Networks. At QCon London 2019 he gave a talk about avoiding being in the news by investigating near misses.
-
Google Releases Google-Landmarks-V2, a Large-Scale Dataset for Landmark Recognition & Retrieval
Google has released Google-Landmarks-v2, an improved dataset for Landmark Recognition & Retrieval, along with Detect-to-Retrieve, a Tensorflow codebase for large-scale instance-level image recognition. Two companion Kaggle competitions based on Google-Landmarks-v2 were also launched. With over 200,000 landmarks in 5 million images, it is the largest landmark dataset ever published.
-
PyTorch 1.1 Release Improves Performance, Adds New APIs and Tools
Facebook AI Research announced the release of PyTorch 1.1. The latest version of the open-source deep learning framework includes improved performance via distributed training, new APIs, and new visualization tools including native support for TensorBoard.
-
Amazon Updates SageMaker Ground Truth with New Labeling Features, Vendor Support and Availability
Amazon announced that SageMaker Ground Truth now offers simplified labeling workflows, support for additional labeling vendors, and is available in the Asia Pacific (Sydney) AWS region – bringing the total to six supported AWS regions in the Americas, Europe, and Asia.