How to Monitor Snowflake Usage and Spend With Datadog

Snowflake is a cloud-based data warehousing service that offers both relational database (SQL) and data warehouse as a service (DWaaS) functionality. While it is a great platform for data warehousing and analytics, it can be challenging to monitor Snowflake usage. That's where Datadog comes in.

Datadog is a monitoring and analytics platform that provides comprehensive visibility into your Snowflake environment. 

It integrates with Snowflake to collect a wide range of data, including query throughput and credit usage. This data is then used to generate detailed graphs and dashboards that give you insights into your Snowflake environment. In this article, we'll look into some ways you can use Datadog to get the most out of Snowflake and its features.

Monitor Credit Consumption

To ensure your organization is getting the most out of its Snowflake investment, it's important to monitor resource consumption and credit usage. By understanding how these two factors impact your overall costs, you can make informed decisions about how to optimize your usage.

Snowflake warehouses only consume credits when they're active. You can use Datadog to see how many credits a warehouse is using by checking the (snowflake.billing.virtual_warehouse.sum). 

If you notice an unusual spike, it might be worth investigating further by looking at the query throughput (snowflake.query.total) over time. You may discover a warehouse is only executing one query every thirty minutes, in which case you may want to decrease its auto-suspend value. You might also want to redirect that query to a more active warehouse and delete the idle warehouse. 

Datadog makes it easy for you to monitor usage across accounts, organizations and even warehouses. 

Optimize Query Performance

Besides suspending and deleting warehouses, you can also resize them at any time, even as they're running, to make sure they perform at the desired level. 

The size of a warehouse determines the speed at which it can perform queries, its memory capacity as well as credits per second it consumes. When a warehouse lacks resources to execute a query, it places it in a queue. Similarly, when it lacks memory to run an operation, it spills data to local and remote storage. 

Datadog logs the daily amount of bytes spilled to local and remote storage, allowing you to gauge whether a warehouse is large enough to handle its workload. 

You can also set alerts that will notify you when a warehouse is under query load and either resize the warehouse or increase the number of clusters to autoscale with the demand. 

Monitor Storage Usage

In Snowflake, storage fees are separate from compute costs and are set at a per TB rate you pay depending on your account type, region and cloud provider. 

Datadog lets you track bytes stored in different Snowflake locations and across different accounts. When looking at database storage used per account, you might want to set an anomaly detector to notify you of a sudden increase in bytes (snowflake.storage.database.storage_bytes). 

You may find that some of the data stored in that account doesn't need to be maintained. You can change a table or set of tables to be temporary or transient and ensure you don't get charged more than a day’s worth of storage fees.

Final Thoughts

Datadog is helpful when it comes to monitoring your Snowflake usage and making sure you're making the most of the service. 

If you're looking for a tool to help you monitor your Snowflake cost, consider using Finout. Finout gives you a detailed breakdown of Snowflake costs per team, feature and even customer, so you can optimize spend and improve your organization's bottom line.

You can also track your Snowflake usage and spend in Finout’s native Datadog app.