Recently Microsoft announced a new data governance solution in public preview on its cloud platform called Azure Purview. This new service automates the discovery of data and cataloging while minimizing compliance risk and helps customers map all their data, no matter where it resides, to provide an end-to-end view of their data estate.
Azure Purview began as a multi-year internal effort under the code "Project Babylon" to assist in Microsoft's own digital transformation and privacy compliance efforts. The company already had a data catalog service with Azure Data Catalog (ADC) that resembles Purview; however, that service is more focused on metadata management than governance – thus making it hard to comply with data protection regulations such as the European Union's GDPR and California's CCPA. With Purview, Microsoft delivers an Azure service that can help customers understand exactly what data their company has, and provide means to manage the data's compliance with privacy regulations, and derive valuable insights.
Alym Rayani, general manager, Compliance Marketing, wrote in a security blog post on Azure Purview:
Azure Purview builds on the same sensitivity labels and data classification taxonomy in Microsoft Information Protection. By extending Microsoft Information Protection's sensitivity labels with Azure Purview, organizations can now automatically discover, classify, and get insight into sensitivity across a broader range of data sources such as SQL Server, SAP, Teradata, Azure Data Services, and Amazon AWS S3, helping to minimize compliance risk.
Azure Purview consists of three main components:
- A data discovery, classification, and mapping component will automatically find all of an organization's data on-premises or in the cloud and evaluate its characteristics and sensitivity.
- A data catalog component enabling users to search for trusted data using a simple web-based experience – the visual graphs let users quickly see if data of interest is from a trusted source.
- And a data governance component provides users a bird's-eye view of a company's data landscape, enabling data officers to efficiently govern data use. This view enables critical insights such as data distribution across environments, how data is moving, and where sensitive data is stored.
Furthermore, since the data map is exposed as an Apache Atlas open API, customers can programmatically push any metadata in lineage from any data system to expand it.
Source: https://www.youtube.com/watch?v=27bA4KFiEKk (screenshot)
Maarten Goet, director at Wortell and Microsoft MVP/RD with expertise around Microsoft Cloud & Enterprise Security technologies, told InfoQ:
I love how Azure Purview can discover and catalog data. Not only from Azure, but also from third party data stores like Amazon's S3 and on-premise SQL. As environments grow and change over the years we've all seen how data spreads out and how hard is it to keep track. Having a service where you can specify either in technical or business terms who has access to what (sensitive or privacy-related) data is a gamechanger!
Also, Mike Flasko, partner director of program management at Microsoft, mentioned the benefit of Azure Purview in a Microsoft Mechanics video:
We're really solving for data discovery and understanding - and providing the foundations for effective data governance, because ultimately, the better you understand the data you have, the more effectively you can use it across your organization.
Microsoft will compete with AWS, which recently announced a partnership with Alation, an independent data catalog provider. AWS will use their service to search & discover and govern data across AWS services, including Amazon Redshift, Amazon EMR, Amazon Simple Storage Service (Amazon S3), AWS Glue, Amazon Relational Database Service (Amazon RDS), and Amazon Athena.
Currently, Azure Purview is available in a few Azure regions such as Southeast Asia, West Europe, and East US. Furthermore, the service is free for customers to try until January 2021, and its details are available on the document landing page.