How To Reduce Logging Costs in GCP

On the Google Cloud Platform (GCP), Cloud Logging is a dedicated service that collects, organizes, analyzes, and monitors your log data, making it easier to troubleshoot and monitor applications, infrastructure, and services.

Though important, logging can come at a steep price, especially if you don't continually assess its volume and retention period. 

In this article, we'll take a look at how Google prices its logging service and discuss some ways in which you can reduce your logging costs. 

This is one of our most popular articles. Don’t miss another highly viewed piece on What is Datadog—read it here.

Table of Contents

1. How Much Does GCP Logging Cost?
2. 5 Ways to Reduce Logging Costs in GCP
3. Reducing Cloud Logging Costs

How Much Does GCP Logging Cost?

Like other cloud services , GCP Cloud Logging costs can contribute significantly to your total cloud spending. 

GCP Cloud Logging pricing is based on how many logs you ingest and the duration of their retention.

• Logging ingestion occurs when log data is sent to GCP Cloud Logging through a log agent or API, with the total cost of log ingestion depending on the volume of log data sent to GCP Cloud Logging. As of April 2023, the price is $0.50 per GB of logs ingested. The first 50 GB of all ingested logs are free per month for each project.
• Logging storage is the amount of data stored and its retention period. As expected, longer retention periods result in higher costs, but may be necessary in order to meet compliance regulations. As of April 2023, the cost is $0.01 per GB per month for logs retained more than 30 days, billed monthly. Logs retained for the default retention period do not incur a storage cost.

Source

Let's use an example to better illustrate how GCP Cloud Logging pricing works. If you generate 500 GB of log data monthly from a web application, the ingestion cost would be $250 per month ($0.50 per GB) minus the free allotment of 50 GB. 

To retain the data for 90 days, you would be charged $0.01 per GB per month for the remaining 60 days, resulting in a monthly storage cost of $45 for 450 GB of log data. The total cost for GCP Cloud Logging in this scenario would be $295 per month.

5 Ways to Reduce Logging Costs in GCP 

There are many ways in which you can reduce or optimize your GCP Cloud Logging spending, which includes setting up exclusions for logs that are not relevant or redundant, and creating log sinks to route logs to specific destinations.

You can even employ more advanced cost control strategies such as fine-tuning retention policies based on log type and frequency of access, and using custom metrics to monitor and alert on unusual log patterns.

1. Set exclusion filters

One of the best places to start from is setting exclusion filters.You can use GCP Console to exclude specific logs from being ingested into GCP Cloud Logging. To set up an exclusion filter, navigate to Logging > Logs Router and click "Create Sink" to create a new sink. In the "Create sink" dialog, give your sink a name and select a destination.

If you already have a sink in place, you can edit it to add the exclusion filter. After that, go ahead and “choose logs to filter out of sink”. From there, create an exclusion filter with the logs you want to ignore. For instance, you could exclude logs from a specific resource, logs with a specific log level, or logs containing specific text. You can even specify the conditions for the logs you want to exclude, since the filter is written in Logging Query Language.

```

resource.type="gce_instance"

AND

resource.labels.instance_id="INSTANCE_ID"

```

With regular reviews and updates of these exclusion filters, you can ensure that you are only ingesting those logs that provide valuable insights into your cloud infrastructure and applications.

2. Export logs to a cheaper storage provider

If you're spending a lot on GCP Logging, consider switching to another provider. Simply configure a Log Sink in your GCP Console and route the log data to another service such as BigQuery. 

In addition to the cost savings associated with exporting logs to cheaper storage options, you can also leverage the powerful analytics offered by these services to gain deeper insights into your log data. 

3. Set log retention periods

Consider the retention period for logs stored in Cloud Logging. Logs are typically retained for 30 days and don't incur storage costs during this period. However, retaining logs for longer than the default retention period can result in increased storage costs.

To avoid unnecessary costs, adjust your retention policies to delete logs that are no longer needed as soon as possible.

4. Use log-based metrics

In GCP Cloud Logging, you can cut costs and reduce the need for long-term storage and processing by creating metrics from logs. You can aggregate and summarize log data, which reduces the amount of raw log data to store and process, particularly if you generate a high volume of log data.

This also means you can customize your log data based on your specific operational needs. For example, you can create metrics that track specific application events or errors, or even metrics that offer insights on infrastructure performance.

5. Use resource-based IAM policies

Resource-based IAM policies restrict access to logs in GCP Cloud Logging. By applying these policies, you can reduce the need for expensive log auditing and ensure that only authorized users can access your logs. This can help you meet compliance and regulatory requirements while also reducing your overall costs. 

Reducing Cloud Logging Costs

Logging can be expensive, especially when logging large volumes of data and retaining them for a long time. Logging is also difficult to track, which is why tools like Finout make it easy to monitor log spending and identify areas for improvement. 

If GCP logging costs are straining your budget, book a demo with one of our specialists today to see how we can optimize your cloud spending.