About SRGTechnical ArticlesRecruitment Information
About SRGTechnical ArticlesRecruitment Information
HOME/Technical Articles/OPA Use Cases in Terraform

OPA Use Cases in Terraform

Ishikawa Kumo (Service Reliability Group (SRG) of the Media Headquarters)@ishikawa_kumo)is.
#SRG(Service Reliability Group) is a group that mainly provides cross-sectional support for the infrastructure of our media services, improving existing services, launching new ones, and contributing to OSS.
This article is a compilation of a previous post on Qiita titled "OPA Use Cases in TerraformThis is an expanded version of the article "
 

overview

Open Policy Agent (OPA) has become popular in recent years for JSON and YAML-based Infrastructure as Code (IaC) management, and in particular, the implementation of OPA and GateKeeper for Kubernetes manifests is becoming standard.
On the other hand, integration with tools like Terraform, which excels at describing complex cloud infrastructure, is still challenging, and there are few examples of actual integration. While there are use cases in combination with Terraform Cloud and Styra, there are not many uses of OPA in pure Terraform environments (for example, when the backend is S3).
In this article, we will focus on the use cases of OPA in a Terraform environment and how to use it in practice.

About OPA and Rego

OPA is an engine for centrally managing Rego policies. It allows consistent policy enforcement across multiple systems for input data in JSON format. Rego is a domain-specific language for OPA, used to express policies over data.
As a declarative programming language, Rego is difficult to use for complex logical processing and dynamic JSON structures. Furthermore, developing Rego policies is significantly different from procedural programming languages, and it is sometimes necessary to write policies from a mathematical logic perspective.

Existing Rego policies and related tools

If you start developing Rego without a development system in place, you may end up with a heavy burden in terms of maintenance. We recommend that you use Rego that has already been created by the community whenever possible.
  • Rego Library Examples of policies shared by the OPA community
For best practices on Rego, it's a good idea to refer to the resources provided by the OPA development company.
It is also an excellent linter.regalis also available, so please try using it.

Rego Notes

Below we have compiled a list of common issues and recommendations for developing Rego policies.
  1. You cannot reassign a variable
      2. name
      sample
  1. Safety Principles
      2. Safety: every variable appearing in the head or in a builtin or inside a negation must appear in a non-negated, non-builtin expression in the body of the rule.
      In short, all variables appearing in the red box below must have a clear, "non-negative" definition.
      not p[x] == 0
      sample
      true
      x
      rego_unsafe_var_error: var x is unsafe
      x
  1. Setting default values
      2. Undefined

Input Selection

OPA validation is JSON-based, so for each Terraform validation target, it must be output as JSON.
Terraform generally verifies the following types of things:
  • state JSON
      • tfstate
  • terraform HCL
      • hcl2json

Policy Evaluation Engine Selection

tfsec・trivy

The tfsec cli has the ability to deploy custom policies written in Rego.
For example, if you implement a simple rego on the following S3, you will get the following results:
Terraform
Rego
tfsec --print-rego-input | jq '.google'
From February 2023 onwards,tfsec is now part of the trivy projectTherefore, it is better to use trivy directly in the future. According to the documentation, trivy also supports custom policies,A more streamlined formatIt can be defined in the metadata.
v0.49.1

tflint

tflint has a Custom Plugin feature. It requires development in the Go language. The following repository is a plugin for evaluating OPA Rego via a Go plugin.
0.5.0
Advance preparation
  1. .tflint.hcl
    1. Preparing the policy
        2. .tflint.d/policies/bucket.rego
        policies
    1. Overall directory structure
      execution
      When running the command, be aware of the following environment variables:
      • TFLINT_OPA_POLICY_DIR
        • policy file
      • TFLINT_OPA_TRACE
        • OPA Tracing
      • TFLINT_OPA_TEST
        • OPA Test Mode
      • TFLINT_LOG=debug
        • TFLINT's own log and OPA Print debug are also available.

      conftest

      conftest has some advantages, such as being able to load HCL directly, and is easier to set up and configure than tfsec or tflint.
      conftest test *.tf -p policies/
      Conftest can also directly use policies published on Github.
      However, Conftest currently only supports rego's trace mode, which may make developing OPA policies difficult.

      Terraform OPA Integration Use Cases

      Defining and enforcing organization-specific policies

      Organization-specific policies, such as tagging for statistics, formatting for multiple environments, and IAM assignment for specific resources, can be flexibly configured using OPA. These policies should be carefully considered based on the actual requirements of the organization.
      For example, there are the following examples:
      All variables are mapped to Terraform Workspaces.
      sample

      Consistency Check

      Validation
      Of course, in most cases, I think it can be solved by redesigning the variable configuration and combining it with validation blocks, pre/postcondition blocks, etc. Rego is suitable for writing complex conditional controls.
      Here are some examples:
      Setting conditions that affect two variables
      sample
      In addition, you can also write logic that you previously checked visually in Rego.
      • Are the Subnet and RouteTable settings of AWS Network Firewall correct?
      • Whether AWS WAF rules implement the required rules

      Part of the automation flow

      Using it as a JSON parser may be a bad idea, but leveraging OPA as a JSON parser in addition to enforcing company-specific policies is effective.
      However, at this time, OPA's output has limited use beyond debugging and error messages, so we recommend evaluating specific compliance policies and rules within your CI flow and using only boolean (True/False) results.
      Here are some examples:
      sample
      If a resource is about to be deleted, this policy will generate an appropriate warning message, prompting the user to double-check if the action is intentional.

      Conclusion

      The integration of OPA and the Rego language has great potential to streamline cloud infrastructure management using Terraform. However, the development of Rego requires considerable effort, which has been one of the barriers to adopting OPA in Terraform environments. Through this article, we hope to have helped you understand the importance of integrating Terraform and OPA and the best practices for doing so. Currently, there are few actual examples of Terraform and OPA integration, but future developments in this area are very promising.
       
      SRG is looking for people to work with us. If you're interested, please contact us here.
       
      SRG runs a podcast where we chat about the latest hot topics in IT technology and books. We hope you'll listen to it while you work.