About SRGArticlesRecruitment Information
About SRGArticlesRecruitment Information
HOME/Articles/OPA Use Cases for Terraform

OPA Use Cases for Terraform

Media Headquarters Service Reliability Group (SRG)@ishikawa_kumo)is.
#SRG(Service Reliability Group) mainly provides cross-sectional support for the infrastructure of our media services, improving existing services, launching new ones, contributing to OSS, etc.
This article is a reprint of a previous Qiita post titled "OPA Use Cases for Terraform" This is an expanded version of the article.
 

overview

Open Policy Agent (OPA) has become popular in recent years for JSON and YAML-based Infrastructure as Code (IaC) management, and in particular the implementation of OPA and GateKeeper for Kubernetes manifests is becoming standard.
On the other hand, integration with tools that excel at describing complex cloud infrastructure, such as Terraform, is still challenging, and there are few examples of actual integration. Although there are use cases in combination with Terraform Cloud and Styra, there are not many uses of OPA in pure Terraform environments (for example, when the backend is S3).
In this article, we will focus on the use cases of OPA in a Terraform environment and how to put it to practical use.

About OPA and Rego

OPA is a centralized engine for managing Rego policies. It allows consistent policy enforcement across multiple systems on input data in JSON format. Rego is a domain-specific language for OPA and is used to express policies over data.
As a declarative programming language, Rego is difficult to process complex logical operations and dynamic JSON structures. In addition, Rego policy development is significantly different from procedural programming languages, and sometimes it is necessary to write policies from the perspective of mathematical logic.

Existing Rego policies and related tools

If you start developing Rego without a solid development structure, you may end up with a heavy burden in terms of maintenance. We recommend that you use Rego that has already been created by the community as much as possible.
  • Rego Library Examples of policies shared by the OPA community
For best practices on Rego, check out what the OPA development company has to offer.
It is also an excellent linter.regalis also available, so please try using it.

Rego Notes

Below we have summarized some common issues and recommendations for developing Rego policies.
  1. You can't reassign a variable
      2. name
      sample
  1. Safety Principles
      2. Safety: every variable appearing in the head or in a builtin or inside a negation must appear in a non-negated, non-builtin expression in the body of the rule.
      In short, all variables appearing in the red box below must have a clear, "non-negative" definition.
      not p[x] == 0
      sample
      true
      x
      rego_unsafe_var_error: var x is unsafe
      x
  1. Setting default values
      2. Undefined

Selecting Input

OPA validation is JSON-based, so for each Terraform validation target, the output must be JSON.
Terraform generally validates the following types of things:
  • state JSON
      • tfstate
  • terraform HCL
      • hcl2json

Policy Evaluation Engine Selection

tfsec・trivy

The tfsec cli has the ability to deploy custom policies written in Rego.
For example, if we apply a simple rego to the S3 below, we get the following results:
Terraform
Rego
tfsec --print-rego-input | jq '.google'
From February 2023 onwards,tfsec is now part of the trivy projectTherefore, it is better to use trivy directly in the future. According to the documentation, trivy also supports custom policies.A more streamlined formatThis can be defined in the metadata.
v0.49.1

tflint

tflint has a Custom Plugin feature. It requires development in the Go language. The following repository is a plugin for evaluating OPA Rego via the Go plugin.
0.5.0
Advance preparation
  1. .tflint.hcl
    1. Preparing the policy
        2. .tflint.d/policies/bucket.rego
        policies
    1. Overall directory structure
      execution
      When executing, please pay attention to the following environment variables.
      • TFLINT_OPA_POLICY_DIR
        • Policy File
      • TFLINT_OPA_TRACE
        • OPA Tracing
      • TFLINT_OPA_TEST
        • OPA Test Mode
      • TFLINT_LOG=debug
        • TFLINT's own log and OPA Print debug are also available.

      conftest

      conftest has some advantages, such as being able to load the HCL directly, and is easier to set up and configure than tfsec or tflint.
      conftest test *.tf -p policies/
      In addition, conftest can directly use policies published on Github.
      However, Conftest currently only supports rego trace mode, which can make developing OPA policies difficult.

      Terraform OPA integration use cases

      Defining and enforcing organization-specific policies

      Organization-specific policies, such as tagging for statistics, formatting for multiple environments, IAM assignment to specific resources, etc., can be flexibly configured using OPA. These policies should be carefully considered according to the actual requirements of the organization.
      For example, there are the following examples:
      All variables are organized in a map that can be distinguished by Terraform Workspace.
      sample

      Consistency check

      Validation
      Of course, in most cases, you can solve the problem by redesigning the variable configuration and combining it with validation blocks, pre/postcondition blocks, etc. Rego is a good way to write complex conditional controls.
      Here are some examples:
      Setting conditions that affect two variables
      sample
      In addition, you can also use Rego to write the logic that you have been checking visually.
      • Are the Subnet and RouteTable settings of the AWS Network Firewall correct?
      • Do AWS WAF rules implement the required rules?

      Part of the automation flow

      Using it as a JSON parser may be a terrible idea, but in addition to enforcing company-specific policies, leveraging OPA as a JSON parser is effective.
      However, at this time OPA has limited use of its output for anything other than debugging and error messages, so we recommend evaluating specific compliance policies and rules within your CI flows and using only the boolean (True/False) results.
      Here are some examples:
      sample
      If a resource is scheduled to be deleted, this policy will generate an appropriate warning message to prompt the user to double-check if the action is intentional.

      Conclusion

      The integration of OPA and the Rego language has great potential to make cloud infrastructure management with Terraform more efficient. However, the development of Rego requires a considerable amount of effort, which is one of the barriers to the adoption of OPA in Terraform environments. Through this article, we hope you have understood the importance of integrating Terraform and OPA and the best practices for doing so. Currently, there are still few actual examples of Terraform and OPA integration, but future developments in this field are very promising.
       
      SRG is looking for people to work with us. If you are interested, please contact us here.
       
      SRG runs a podcast where we chat about the latest hot topics in IT and books. We hope you will enjoy listening to it while you work.