Matt Winkler delivered a talk at Microsoft Build 2018 explaining what is new in Azure Machine Learning.
The Azure Machine Learning platform is built from the hardware level up. It is open to the tools and frameworks of your choice. If it runs on Python, you can do it within the tools and frameworks. Services come in three flavors: conversational, pre-trained, and custom AI.
It is a fully managed platform. You can build, deploy, and manage models at scale. The preparation of data, the building and training of models, as well as their deployment enables agile development. This reduces the latency between the idea and when it appears in the world. It is also necessary because there is no guarantee of success with your models, or that patterns will be found. You can deploy AI everywhere, to the cloud, on premise, in a remote facility with no internet connectivity, or to edge devices such as tractors, sensors, or cameras. You also get telemetry data to help you refine your model.
The new improvements come in several areas: making development easier, single container deployment to make the dev/test loop faster, using the SDK from the Azure Notebook for control, as well as helping people get started solving a particular problem.
To solve these problems, Microsoft has put several items in preview today: Project Brainware hardware accelerated models, Azure ML packages, for vision, text, and forecasting, the ONNX model gallery, an updated SDK to improve Azure notebooks, hyperparameter turning services, the addition of ACI/AKS batch AI compute targets, and IOT deployment enhancements.
Packages are common tasks or approaches to vision, text, or forecasting that are not found in the frameworks. They are Python packages that can be deployed on Azure ML. You can use the defaults, or get a fair amount of control over the training.
Hardware accelerated models use FPGAs to drive amazing enhancements to performance. This enables real time scoring so you can do incremental learning as new data comes in. For example, you can score a single image in 1.8 milliseconds. This can give you real time AI at cloud scale with industry leading performance at low cost.
The ONNX Model Gallery is as set of already developed models that you can use to avoid the time and expense of building models where it is not necessary. You can then skip the build and train step of model development. The common interchange format ONNX allows you to use the models in various places such as inside an application, or to deploy it to services.
The SDK improvements give improvements to Azure Notebooks. These can be installed anywhere you can run Python. These improvements make it easier to provision compute targets, run training jobs local on the notebook, or a large scale cluster. It incorporates hyperparameter tuning, built based on internal Microsoft company experience. You can also register and deploy models.
Hyperparameter tuning allows you to find the right configuration for the right model. You can define the parameters you want to search, and how you search (grid based, random). There is also an early termination policy if you can detect that the model you are working on is not going to be good. You can then avoid spending money on fruitless search. Microsoft has found that at scale, you can reduce compute expense by about 50%. You can then explore more space for the same money.
Microsoft has a partnership with Qualcomm on a developer kit for vision hardware acceleration so that you can deploy models trained with Azure ML. There is also an Azure AI toolkit for IOT Edge. It is designed for scenarios such as a retail register processing receipts, or tractors in the field. These are scenarios without IT infrastructure.