Infra CI Pipeline Design

In today's edition, I want to talk about CI pipelines for Infrastructure as Code.

We will look at,

1. Infra Code Repository

2. Infra Code Git Branching Strategy

3. Pull Request Based Workflow

4. Design a Infra CI Pipeline (Practical Example)

No one needs an introduction to Infrastructure as Code (IaC) because it has become the standard for managing infrastructure across small, medium, and large organizations.

But does everyone treat it with the same importance as application code?

Application code goes through multiple levels of testing, validation, and approvals in a pull request (PR)-based workflow.

IaC should be treated the same way. It needs to follow a Git-based PR workflow with automation, reviews, and validation for every change.

Infra Code Repository

All IaC code should be part of a Git repository.

It could be written in Terraform, Ansible, Kubernetes manifests, Helm charts, etc.

These repositories could be managed by individual project teams or a central platform team.

Here is a simple Terraform repository structure that includes variables for different environments.

Who is responsible for adding or modifying this code?

It depends on the project. In some cases, DevOps engineers maintain the entire infrastructure code. Sometimes, platform teams manage the core infrastructure code, while DevOps teams import core modules and write wrappers to customize them as needed.

I have also seen cases where the networking team manages its own Terraform modules to handle cloud networking.

Similarly, in some projects, developers make necessary changes to the infrastructure code.

Ultimately, it depends on how an organization or project is structured. There is no single workflow for managing it.

But regardless of who manages it, how do you ensure that every change is properly tested?

This is where the Infrastructure CI pipeline comes into play.

Infra Code Git Branching Strategy

To have a well-defined infrastructure development and testing pipeline, you need a branching strategy.

For infrastructure pipelines, the branching strategy is usually quite simple.

A trunk-based branching strategy simplifies the branching model by focusing on a single main branch where all changes are merged frequently.

Short-lived feature branches are created for new changes and hotfixes and are merged quickly.

Typically, these branches are named using task IDs. For example, a Jira story ID like CR-135. This approach is common because Jira offers Git integration, allowing you to track Git commits directly within the Jira story.

Here is an example.

Release snapshots (or versioned snapshots) are very useful for infrastructure CI pipelines, especially during version upgrades.

Release snapshots (or versioned snapshots) can be implemented using Git tags with semantic versioning (MAJOR.MINOR.PATCH format - e.g., v1.2.0).

For example,

git tag v1.2.0
git push origin v1.2.0

Pull Request Based Workflow

Infrastructure code should be treated the same way as application code.

This means that every change in the infrastructure code should go through Pull Requests (PRs).

Why?

1. Pull requests allow all changes to be tracked and reviewed, ensuring that every modification is documented and accountable. You get a clear history of who did what and why.

2. Pull requests can trigger automated tests to validate the infrastructure configuration before deployment, say, to validate a Terraform plan or check compliance before anything goes live.

3. If issues arise after a deployment, changes can be easily reverted to a previous version since all modifications are tracked and versioned

4. Also, it ensures that changes follow a standardized process, similar to application code.

The key idea here is risk management.

Infrastructure affects uptime, security, and cost, so skipping reviews is like skipping seatbelts 😀 . It might seem fine until something goes wrong.

Lets Design a Infra CI Pipeline

Let's design an Infrastructure CI pipeline for Terraform code to put everything we've discussed into practice.

Following are the goals for the Terraform CI Pipeline on every PR raised.

1. Lint and Validate: Ensure Terraform code is syntactically correct and follows best practices using tools like tflint.

2. Security and Compliance Scan: Detect potential security or compliance issues in the Terraform code using tools like checkov

3. Plan: Preview changes to catch unintended consequences. Share the output plan (e.g., as a PR comment or artifact) for visibility.

4. Testing (Optional): Simulate or verify configurations where possible using tools like Terratest

5. Review: Apply changes safely with approval gates. Require manual approval from a teammate after reviewing the terraform plan output.

The following image illustrates the end-to-end Infrastructure CI pipeline workflow based on a Pull Request (PR) process, aligned with the goals outlined above.

I have used Jenkins as an example, but this workflow applies to any CI tool like GitLab CI/CD, GitHub Actions, or CircleCI. The key steps and logic remain the same across different platforms.

(Open the image in new tab for HD)

Tomorrow

One of the bad practices I have seen in IaC management is using poor Git commit messages.

In tomorrow's edition, I will share some best practices for writing commit messages and explain why it's important.