Google brings Cloud Monitoring metrics in BigQuery as a new capability in preview to combine billing data with resource utilization metrics. The combination allows users to perform detailed cost analyses in BigQuery. In addition, the company provides an out-of-the-box Looker Studio template that combines Cloud Monitoring and detailed billing metrics for Google Kubernetes Engine (GKE).
According to the company, Cloud Monitoring metrics in BigQuery offer Google Cloud users advantages such as identifying and optimizing resource allocation, enabling cost-driven decision-making for workload scaling and infrastructure optimization, and providing enhanced cost visibility by uncovering hidden costs and eliminating unnecessary expenses. Furthermore, it improves chargeback visibility by accurately allocating costs to individual teams or projects, promoting fair and transparent chargeback practices.
Integrating resource metric data and billing data gives customers better visibility of costs. The authors of a Google blog post write:
By joining billing data with GKE resource usage metrics, you can now attribute specific costs to individual workloads, namespaces, or even pods within their clusters. This granular level of detail empowers you to pinpoint the sources of your cloud expenditures and make data-driven decisions to optimize resource allocation and minimize costs.
Richard Seroter, chief evangelist at Google Cloud, states in his daily reading list blog post:
Combining billing data with utilization data for Kubernetes clusters? That’s a big deal, and should change the game for those figuring out cluster costs.
Overview GKE Spend (Source: Google Cloud blog post)
The preview capability allows users to generate granular cost insight for GKE by joining Google Cloud Observability Analytics data and billing data.
In a LinkedIn post by Joy Wang, a Google product manager, and Vipul M., a SRE leader and Equifax Fellow, commented:
Why just GKE? How about all GCP products resource metering combined with billing and query using labels enriched with meta data to drive insights that give deeper context insights?
With Wang responding:
Great suggestion! More will come!
Cloud providers like Microsoft natively offer several services allowing users to consolidate billing data and resource utilization metrics, yet they do not have Google's current preview capability. In Azure, to combine billing data and resource utilization for the GKE counterpart Azure Kubernetes Service (AKS), users must get billing data from the Azure Cost Management and Billing tool and Azure Monitor and Azure Log Analytics for metrics.
Lastly, the capability is in preview; customers must sign up for access.