About SRGTechnical ArticlesRecruitment Information
About SRGTechnical ArticlesRecruitment Information
HOME/Technical Articles/Easily notify GKE cluster upgrades via Slack

Easily notify GKE cluster upgrades via Slack

This is Kobayashi (@berlinbytes) from the Service Reliability Group (SRG) of the Media Headquarters.
#SRG(Service Reliability Group) is a group that mainly provides cross-sectional support for the infrastructure of our media services, improving existing services, launching new ones, and contributing to OSS.
This article is a report on our verification of how it is now possible to easily notify GKE cluster-level events to Slack.
 

Introduction

Until now, GKE cluster-level events were only published to Cloud Pub/Sub, so if you wanted to receive notifications on Slack,
  • GKE -> Cloud Pub/Sub -> Cloud Functions -> Slack
  • GKE -> Cloud Pub/Sub -> Datadog Logs -> Slack
We needed a mechanism to trigger events on topics received from Pub/Sub, such as:
 
That will start from around February 2025.GKE UpgradeEvent notificationIt seems that notifications are now also being sent to Cloud Logging, so I think it will be easier than ever to send notifications to Slack, so I will be testing it out.
 

Set up notifications

This can be easily achieved by triggering a log-based alert policy when a log is sent from GKE to Cloud Logging and then sending that alert to Slack.
 
 

Add Slack to your notification channels

First, set up a notification channel for Slack. The documentation says that you can add this when creating an alert, but an error occurs with the redirect when setting it up, so we'll show you how to avoid this by preparing it in advance.
  1. Edit notification channels
  1. Add New
  1. 許可する
  1. If you do not have administrator privileges, an additional request will be sent to the administrator. If you have the privileges, click Next.
  1. Slack Channel Name
  1. テスト環境通知用
  1. テスト通知を送信
 

Creating an alert policy

Let's set up a log-based alert policy.
  1. Log ExplorerAccess
  1. In the query pane, create a query like this one that suits your environment:
    1. クエリを実行
    1. If there is no output, expand the period of the log and check that there is output.
    1. ログアラートの作成
    1. ログベースのアラート ポリシーの作成
      1. GKE UpgradeEvent Notification
      2. No Severity
      3. 次へ
      4. Preview logs
      5. 次へ
    1. Slack
    1. Save
     

    Test the alert

    We will manually write and publish log entries and verify that alerts are triggered.
    1. Method: entries.write  |  Cloud Logging  |  Google CloudAccess to
    1. Request body
      1. Execute
      1. If prompted, follow the instructions for the authentication flow
      1. 200
       
      Now you can notify Slack about GKE upgrades.
      That was easy!

      price

      There was a lot of talk about the announcement that the service would become paid from this year, but as of March 2025, the alert policy is free.
      The page below states that charges will begin in April 2026.
      • $1.50 per month per condition in the alert policy.
      • $0.35 per 1,000,000 time series returned by a query in a metric alert policy condition.
      月額 $1.50
       

      Conclusion

      Being able to easily send notifications to Slack without having to process Pub/Sub was easy to use and very convenient.
      Other cluster-level notifications include:
      It seems that this is now also output to Cloud Logging.
      It is also useful to set up notifications on Slack using the same procedure.
      It's also easy to send notifications via email, SMS, and PagerDuty!
       
      SRG is looking for people to work with us. If you're interested, please contact us here.