Microsoft Releases Data Science Tools for Interactive Data Exploration and Modeling

| by Srini Penchikala Follow 35 Followers on Nov 07, 2016. Estimated reading time: 1 minute |

Microsoft recently released two new data science tools for interactive data exploration: modeling and reporting. These data science utilities, called Interactive Data Exploration, Analysis and Reporting (IDEAR) and Automated Modeling and Reporting (AMAR) can be reused by data science teams for specific tasks in their projects.

Data science teams spend a significant amount of time writing code to answer data related questions like data schema, missing data elements, individual variable distribution & transformation, specific clustering patterns in the data, and the performance of Machine Learning (ML) models. These two tools can be used to automate these common tasks in the data science lifecycle. The goal is to ensure consistency and completeness of data science tasks across different projects in the organization.

Interactive Data Exploration:

IDEAR tool can be used to explore, visualize and analyze data and provide insights into the data. Based on Shiny library from R Studio, IDEAR includes data export and report generation features. Data export includes saving the associated R scripts generating the visualization to a R log file. The users can run the R log file to generate the data report automatically.

Other features of IDEAR include automatic variable type detection, variable ranking and target leaker identification, and visualizing high-dimensional data.

Automated Modeling and Reporting:

AMAR is used to train machine learning models with hyper-parameter sweeping, compare the accuracy of the models and assess the variable importance. We specify in a parameter input file, the ML models to run, which data to use for training and testing, the parameter ranges to sweep over, and the strategy for best parameter selection.

The model report generated by AMAR tool contains the model information, model evaluation & comparison and also the feature ranking.

IDEAR and AMAR tools run in CRAN-R and can be accessed from the GitHub website. This repository is part of the Team Data Science Process (TDSP) which was launched at Microsoft Machine Learning & Data Science Summit last month.

If you are interested in learning more about these data science tools, check out Microsoft Technet blog post or the Azure TDSP Utilities GitHub website.


Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread


Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you