Policy as Code: A Must-Know Workflow for DevOps Professionals

👋 Hi! I’m Bibin Wilson. In each edition, I share practical tips, guides, and the latest trends in DevOps and MLOps to make your day-to-day DevOps tasks more efficient. If someone forwarded this email to you, you can subscribe here to never miss out!

In the previous edition, we looked at policy engines like Kyverno, OPA, etc. This edition focuses more on how these engines fit into infrastructure management. (Essential Concepts a DevOps Engineer Should Know)

In my personal experience working in infrastructure management (even in large organizations), I have seen several issues, such as:

• Security incidents

• Usage of unverified Docker images in production

• Improper resource tagging, resulting in VMs and clusters running in environments without a valid purpose.

• Difficulty in tracking costs due to non-compliance with required cost center tags.

• Deployment of containers and VMs with unnecessary CPU/memory resources, leading to huge bills

and more…I refer to this as the "Lack of Infrastructure Discipline."

These problems happen because there’s no proper policy enforcement. That’s where governance comes in.

So what is governanece?

It’s about setting rules for how things should work in your infrastructure and making sure everyone follows them.

The best way to enforce these policies is through policy as code (a vital component of DevSecOps)

In this edition, we will look in to,

1. What is a Policy?

2. What is Policy as Code (PaC)?

3. Need for Policy as Code?

4. Policy as Code Work Workflow (With illustration)

What is a Policy?

In literal terms, Policy = Rules.

In the context of IT Infrastructure, policies are rules or guidelines that dictate how infrastructure should be set up, managed, and maintained.

Here are some real world examples of infra policies.

1. Tag Policy: Any cloud resources created have standard tags, such as the app name, environment, cost center details, etc.

2. Image Security Policy: Only approved, scanned and signed container images should be deployed on Kubernetes.

3. Resource Allocation Policy: Development environments should only use small to medium-sized virtual machines to control costs.

4. Resource Quota Policy: Only a “X” amount of CPU, memory, and storage should be allocated to each namespace in a Kubernetes cluster.

Now, who is responsible for creating these policies?

Usually these policies are documented and shared with project teams by the compliance or security teams.

Now that we have an understanding of what 'policy' means in DevOps/IaC, let's look at what 'policy as code' means.

What is Policy as Code (PaC)?

Simply put, Policy as Code is the idea of writing your organization's infrastructure rules and policies in a code format.

Unlike traditional methods that rely on manual processes and documentation, this approach uses code to automate and enforce policies across your IT infrastructure.

Implement PAC is not an easy task. It requires collaboration between different teams for effective policy implementation.

Why Policy as Code?

The manual management of policies often leads to inconsistencies and errors.

For example, when a new engineer joins the team, they must be made aware of existing policies. Without this knowledge, they may develop Infrastructure as Code (IaC) that does not comply with organizational standards.

Take, for instance, a policy that mandates specific tags for every virtual machine.

If an engineer is unaware of this requirement, he/she might deploy virtual machines without the necessary tags, leading to non-compliance.

That is why we need PAC implementation to enforce the policies.

How Does Policy as Code Work?

Policy as Code involves defining policies in a machine-readable format.

Tools like Checkov, Open Policy Agent or Kyverno help interpret and enforce these policies.

For instance, you can write a policy that checks if your cloud configurations meet security standards.

Or write a policy that doesn't allow the use of latest tag in contianer images.

So that question you might have is,

So how do we enforce this policies so that it can check all the IaC code developed by engineers?

The answer is Pull Request Workflow

Pull Request Workflow

All your IaC should have Git Pull Request Workflow.

Then integrate your Policy as Code tool with the PR workflow. It follows shift-left approach, which means enforcing standards and policies early in the IaC development lifecycle.

The following image shows the workflow of policy as code.

Here is how it works

1. Start by defining clear and specific policies for your infrastructure. These policies might include security standards, resource allocation limits, naming conventions, etc. Use a Policy as Code tool like Checkov, Open Policy Agent (OPA), or Kyverno to define these policies.

2. Establish a PR-based workflow for IaC development. Any changes to the infrastructure should be made via a branch, followed by a PR to merge the changes into the main branch.

3. When a PR is created or updated, the policy as code tool automatically checks the proposed changes against the defined policies. This is typically done through a Continuous Integration (CI) pipeline using tools like Jenkins, GitHub Actions, GitLab CI/CD, etc.

4. If the changes pass the policy checks, they move forward to the review stage where team members can review and discuss them. If the changes fail the checks, the engineer is notified to make the necessary changes to comply with the policy.

5. Once the PR is approved and it complies with all policies, it can be merged into the main branch. From there, the changes can be deployed to your infrastructure.

Wrapping Up

You won’t find Policy-as-Code (PAC) implementation in many organizations, even though it is an important practice.

The reasons for not implementing PAC vary, ranging from a lack of awareness to the amount of work and processes required to deploy it at scale.

However, if you start small and gradually implement PAC, many mistakes or security incidents can be avoided. In fact, it can even lead to significant cost savings over time.

Over to you!

What are your thoughts on Policy-as-Code?

Do you think it would benefit your infrastructure management?

Comment below. I’d love to hear your thoughts on this!