About SRGArticlesRecruitment Information
About SRGArticlesRecruitment Information
HOME/Articles/Easily notify GKE cluster upgrades via Slack

Easily notify GKE cluster upgrades via Slack

This is Kobayashi (@berlinbytes) from the Service Reliability Group (SRG) of the Media Headquarters.
#SRG(Service Reliability Group) mainly provides cross-sectional support for the infrastructure of our media services, improving existing services, launching new ones, contributing to OSS, etc.
This article is a report on our verification of how it is now possible to easily notify GKE cluster-level events to Slack.
 

Introduction

Until now, GKE cluster-level events were only published to Cloud Pub/Sub. If you wanted to be notified via Slack,
  • GKE -> Cloud Pub/Sub -> Cloud Functions -> Slack
  • GKE -> Cloud Pub/Sub -> Datadog Logs -> Slack
We needed a mechanism to trigger events on topics received from Pub/Sub, such as:
 
That will start from February 2025.GKE UpgradeEvent notificationIt seems that notifications are now also available in Cloud Logging, so it should be easier than ever to send notifications to Slack, so I'd like to test it out.
 

Setting up notifications

This can be easily achieved by triggering a log-based alert policy when a log is released from GKE to Cloud Logging and then directing that alert notification to Slack.
 
 

Add Slack to your notification channels

First, set up a notification channel for Slack. The documentation says that you can add this when creating an alert, but an error will occur with the redirect when setting it up, so we'll introduce a procedure that avoids this by preparing it in advance.
  1. Edit notification channels
  1. Add New
  1. 許可する
  1. If you do not have administrator privileges, an additional request will be sent to the administrator. If you have the privileges, click Next.
  1. Slack Channel Name
  1. テスト環境通知用
  1. テスト通知を送信
 

Creating an alert policy

Let's set up a log-based alert policy.
  1. Log ExplorerAccess
  1. In the query pane, create a query like this one that suits your environment:
    1. クエリを実行
    1. If there is no output, expand the period of the logs to check that there is output.
    1. ログアラートの作成
    1. ログベースのアラート ポリシーの作成
      1. GKE UpgradeEvent Notification
      2. No Severity
      3. 次へ
      4. Preview logs
      5. 次へ
    1. Slack
    1. Save
     

    Test the alert

    We will manually write and issue log entries and verify that alerts are triggered.
    1. Method: entries.write  |  Cloud Logging  |  Google CloudAccess to
    1. Request body
      1. Execute
      1. If prompted, follow the authentication flow
      1. 200
       
      Now you can notify Slack about your GKE upgrade.
      That was easy!

      price

      There was a lot of talk about the announcement that it would become a paid service starting this year, but as of March 2025, alert policies are free of charge.
      The following page states that charges will begin in April 2026.
      • $1.50 per alert policy term per month.
      • $0.35 per 1,000,000 time series returned by a query in a metric alert policy condition.
      月額 $1.50
       

      Conclusion

      Being able to easily send notifications to Slack without having to process Pub/Sub was easy and convenient.
      Other cluster-level notifications include:
      It seems that it is now also output to Cloud Logging.
      It is also useful to set up notifications in Slack using the same procedure.
      It's also easy to send notifications via email, SMS, and PagerDuty!
       
      SRG is looking for people to work with us. If you are interested, please contact us here.