TEKNOLOGI

When using Bicep modules for multiple projects or for diverse clients, an often straightforward approach is to define the modules centrally and replicating those modules in each repository. While this approach allows customization for specific solutions, such as incorporating new features or implementing security configurations for compliance, challenges arise when attempting to propagate these improvements to other solutions while ensuring adherence to the latest standards.

This is where the private Bicep module registry steps in. At my previous company, we encountered a significant challenge as the scale and complexity of infrastructure projects grew. Copying module files became time consuming and inefficient, resulting in a lot of duplicate code, making it challenging to maintain consistency and keep modules up to date. Improvements where not shared among developer teams and across the whole organization, and handling this manually was error-prone and time comsuming. The private registry really helped us by providing an elegant solution as a centralized repository which has all shared modules in one location. This not only ensures a single source of truth for modules but also simplifies the updates process. When changes are made to a module in the registry, they are automatically reflected in all projects referencing that module, promoting consistency and reducing the risk of discrepancies.

Moreover, the registry approach promotes reusability and collaboration. Teams can easily share and leverage standardized modules without the need for redundant copies in every project. This streamlines the development process, accelerates project timelines, and fosters a collaborative environment where best practices are maintained consistently.

Prerequisites

Implementing a private Bicep module registry requires carefull planning and consideration of several prerequisites to ensure an effective integration for a new infrastructure workflow. These are some key prerequisites which where important for us:

• Access control: By centralizing modules in a secure registry, we can implement fine-grained access permissions ensuring that only authorized teams can access specific modules.
• Governance: Centralized control must be in place which is critical for maintaining the integrity and security of infrastructure.
• Consistency: The registry must improve maintainability and consistency for all teams across the organization.
• Version strategy: A good version strategy must be in place for the Bicep modules. This is crucial for tracking changes, ensuring no breaking changes, and for managing updates across different projects.
• Documentation: Each module must have clear documentation for teams on how to use or contribute to the bicep modules, see the necessary configurations for each version of a module, and when new versions are available.
• Shift left: The Bicep modules must be automatically checked for misconfigurations and best practices as part of the CI/CD process, enabling early detection of issues and adherence to industry standards.
• Collaboration: Teams must be able to contribute to the Bicep module registry with a approval workflow in place.

We started small by first by creating templates and documentation and using the modules for a couple of projects. This way the development teams could also see the benefits and give feedback about the usability and effectiveness of the Bicep modules. With new insights, we iteratively implemented new improvements and optimizations for the Bicep registry. This iterative approach not only allowed us to fine-tune the modules to meet the specific needs of each project but also fostered a culture of continuous improvement within the development teams. As a result, our infrastructure as code practices evolved organically, aligning closely with the dynamic requirements of our projects.

Bicep module registry

A Bicep module registry is a centralized repository where Bicep modules are stored and managed. A central Azure Container Registry is used as a central hub to store and maintain the bicep modules.

For our solution, we are using 2 container registries:

• The main registry that contains the bicep modules that can be used by the organization.
• A staging registry that contains the bicep modules that are being used for testing modules before promoting the modules to main registry.

Each Azure resource type is stored as a repository in the Container Registry. A repository consists of bicep templates and (sub)modules. These templates can be invoked from CI/CD pipelines and/or automation scripts (Azure CLI, Powershell).

All the code and templates that are describe in this post can be found here: teknologi-bicepregistry

Bicep modules

For each Azure resource type or resource a new folder is created, which consists of:

├── <resource type name>
│   ├── bicep
│   │   ├── README.md
│   │   ├── main.bicep
│   │   ├── modules
│   │       ├── <submodule>.bicep
│   │       ├── <submodule>.bicep

An example can be found here

main.bicep

This is the ‘core’ bicep module for deploying the infrastructure, providing the foundational infrastructure and configuration necessary for the entire deployment. The main template can invoke submodules from the modules folder if necessary. Each main module must import a environment configuration file (ENVIRONMENT.json) as config parameters.

submodule.bicep

The main bicep module can contain multiple submodules that are necessary to perform complex infrastructure deployment and modularize the code for better maintainability and reusability.

README.md

Each bicep template must contain a README with the necessary information which explains how to use the template and what configurations are necessary. For consistency all README files must share the same format.

GitHub Actions

For creating and testing multiple bicep modules in a structured manner, we are using a GitHub Actions workflow:

There are a couple of files that we are using for testing the modules and for metadata:

Examples of the files can be found here

With this approach the deployment and configuration can be centralized, which ensures consistency and simplifies the process creating and testing the templates. The deployment files are setup so each user can use the same settings and configurations for quickly creating and changing templates. The deployment files will also be used for automated testing and validation.

There are 2 GitHub Action workflows:

Pull request workflow

This workflow is started when a pull request has been created and has the following steps:

• Checks which module files have been created or updated.
• Get module(s) information from the templates file.
• Get module(s) latest version from the main registry.
• Checks if the README.md documentation for the module(s) are correct.
• Creates ARM parameter file (input from deployment config file) for PSRule for Azure scanning.
• Scans the module(s) with PSRule for Azure for best practices.
• Scans the module(s) with Trivy for misconfigurations.
• Adds or updates a summary about the scans as a comment in the pull request.

Push master workflow

This workflow is started when a pull request has merged with master and has the following steps:

• Checks which module files have been created or updated.
• Get module(s) information from the templates file.
• Publishes the module(s) to the main registry.
• Updates the README.md documentation with the commit sha for the new version (to be able to see the commit code for each module version).

Steps

These are the steps that are necessary to create or update a Bicep module:

Local development

• Create a new branch.
• Write the code for the new or updated bicep template(s).
• Add the code to invoke the main module to the deployment main (main.bicep) (and deployment init (init.bicep) if necessary).
• Add the parameters for the module to the deployment config.
• Add the new bicep module to the template file (templates.jsonc).
• Run the deployment staging Powershell script for deploying the bicep templates to the staging repository and deploy test resources as described.

Promote to production

To promote the new templates to the production bicep registry, these steps are necessary:

Documentation

• Create the README file for the new module.

Pull request

• Create a pull request for the branch to merge with the master branch.
• Several check are included in the pull request workflow, like for example PSRule for Azure, which checks if the modules are created according to best practices.

Merge to master

• After approval, the branch is merged with the master branch. A GitHub Action workflow is triggered that:
  • Deploys the new template to the main registry with a new version.
  • Updates the bicep module README documentation with a link to the GitHub commit that is added to the version in the ChangeLog. This way it is possible to view the committed code for each version, with all necessary code and documentation for troubleshooting.

Pull request summary comment automation

The Pull Request GitHub Action workflow has a step to publish information about the modules that are updated or created as a comment in the pull request. It creates a new comment for each module with information and the scan results from PSRule and Trivy.

To be able to create comments in the pull request, a GitHub App needs to be created that has pull_request: write access on the repository. It uses the following steps:

• peter-evans/find-comment@v3

This GitHub Action step is used to find an existing pull request summary comment.

• peter-evans/create-or-update-comment@v4

This GitHub Action step creates or updates an existing pull request summary comment. When the pull request summary comment already exists, it overwrites the comment with new data.

A GitHub Action example can be found here

A pull request summary comment looks like this:

Repository variablen

GITHUBAPPID

This repository variable contains the Application ID for the GitHub App. This GitHub App is used to create a GitHub App token to authenticate to the GitHub repository.

Security

There are 2 repository secrets that are used by the GitHub Actions workflows:

BICEP_GITHUBACTIONS

This secret contains the secret data for the Azure AD Service Principal to access the Azure subscription where the Container registry resources exists in the following format:

{
  "clientId": "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx",
  "clientSecret": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "subscriptionId": "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx",
  "tenantId": "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx"
}

GITHUBPRIVATEKEY

This secret contains the private key for the GitHub App. This GitHub App is created in the GitHub organization. The GitHub App is added to the master branch protection rule in the repository to bypass required pull requests. This way the GitHub App can commit code in the master branch through an GitHub Action workflow.

In conclusion, implementing a private Bicep module registry was really a big improvement for us and the organization. It helped in streamlining the management and deployment of infrastructure across diverse projects. By centralizing shared modules in a shared repository, the registry not only ensures a reliable source of truth but also simplifies the process of maintaining and updating modules, and improved collaboration and reusability between the development teams.