terragrunt VS terraform-docs

‍Terragrunt is a thin wrapper that provides extra tools for keeping your Terraform configurations DRY (Don't Repeat Yourself), working with multiple Terraform modules, and managing remote state. It's particularly useful in managing large-scale infrastructure deployments with Terraform.

... #************************** Terraform ************************************* ARG TERRAFORM_VERSION=1.7.3 RUN set -ex \ && curl -O https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip && unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d /usr/local/bin/ RUN set -ex \ && mkdir -p $HOME/.terraform.d/plugin-cache && echo 'plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"' > ~/.terraformrc #************************* Terragrunt ************************************* ARG TERRAGRUNT_VERSION=0.55.1 RUN set -ex \ && wget https://github.com/gruntwork-io/terragrunt/releases/download/v${TERRAGRUNT_VERSION}/terragrunt_linux_amd64 -q \ && mv terragrunt_linux_amd64 /usr/local/bin/terragrunt \ && chmod +x /usr/local/bin/terragrunt #*********************** Terramate **************************************** ARG TERRAMATE_VERSION=0.4.5 RUN set -ex \ && wget https://github.com/mineiros-io/terramate/releases/download/v${TERRAMATE_VERSION}/terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && tar -xzf terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && mv terramate /usr/local/bin/terramate \ && chmod +x /usr/local/bin/terramate #*********************** tfsec ******************************************** ARG TFSEC_VERSION=1.28.5 RUN set -ex \ && wget https://github.com/aquasecurity/tfsec/releases/download/v${TFSEC_VERSION}/tfsec-linux-amd64 \ && mv tfsec-linux-amd64 /usr/local/bin/tfsec \ && chmod +x /usr/local/bin/tfsec \ && terragrunt --version #**********************Terraform docs ************************************ ARG TERRRAFORM_DOCS_VERSION=0.17.0 RUN set -ex \ && curl -sSLo ./terraform-docs.tar.gz https://terraform-docs.io/dl/v${TERRRAFORM_DOCS_VERSION}/terraform-docs-v${TERRRAFORM_DOCS_VERSION}-$(uname)-amd64.tar.gz \ && tar -xzf terraform-docs.tar.gz \ && chmod +x terraform-docs \ && mv terraform-docs /usr/local/bin/terraform-docs #********************* ShellCheck ***************************************** ARG SHELLCHECK_VERSION="stable" RUN set -ex \ && wget -qO- "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION?}/shellcheck-${SHELLCHECK_VERSION?}.linux.x86_64.tar.xz" | tar -xJv \ && cp "shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/bin/ \ && shellcheck --version ...

Our first attempt was to introduce other engineering teams to Terraform - the Platform team was already using it extensively with Terragrunt, and using Atlantis to automate plan and apply operations in a Git flow to ensure infrastructure was consistent. We'd written modules, with documentation, and an engineer would simply need to raise a PR to use the module and provide the right values, and Atlantis (once the PR was approved by Platform) would go ahead and set it up for them.

Terragrunt: An extension of Terraform, Terragrunt assists in managing complex infrastructure with less duplication and more efficiency. Its power lies in its ability to manage dependencies and its dry configuration approach.

Copy the link and download on your terminal using the wget command. Example: wget https://github.com/gruntwork-io/terragrunt/releases/download/v0.54.19/terragrunt_linux_amd64

name: Configure on: push: branches: - main pull_request: branches: - main jobs: terraform: runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v2 - name: Setup SSH uses: webfactory/[email protected] with: ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }} - name: Setup Terraform uses: hashicorp/setup-terraform@v2 with: terraform_version: 1.5.5 terraform_wrapper: false - name: Setup Terragrunt run: | curl -LO "https://github.com/gruntwork-io/terragrunt/releases/download/v0.48.1/terragrunt_linux_amd64" chmod +x terragrunt_linux_amd64 sudo mv terragrunt_linux_amd64 /usr/local/bin/terragrunt terragrunt -v - name: Apply Terraform changes run: | cd dev terragrunt run-all apply -auto-approve --terragrunt-non-interactive -var AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -var AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -var AWS_REGION=$AWS_DEFAULT_REGION cd apache-server/ec2-web-server public_ip=$(terragrunt output instance_public_ip) echo "$public_ip" > public_ip.txt cat public_ip.txt env: AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }} - name: Upload artifact uses: actions/upload-artifact@v4 with: name: ip-artifact path: dev/apache-server/ec2-web-server/public_ip.txt ansible: runs-on: ubuntu-latest needs: terraform steps: - name: Download artifact uses: actions/download-artifact@v4 with: name: ip-artifact - name: Configure Ansible run: | sudo apt update sudo pipx inject ansible-core jmespath ansible-playbook --version sudo echo "[web]" >> ansible_hosts sudo cat public_ip.txt >> ansible_hosts mv ansible_hosts $HOME sudo cat $HOME/ansible_hosts - name: Configure playbook run: | cd $HOME cat > deploy.yml < Test Page This is a test page EOF cat $HOME/deploy.yml - name: Run playbook uses: dawidd6/action-ansible-playbook@v2 with: playbook: deploy.yml directory: /home/runner key: ${{secrets.SSH_PRIVATE_KEY}} options: | --inventory ansible_hosts --verbose

Created and maintained by Gruntwork, Terragrunt is a tool designed to enhance Terraform’s capabilities. It acts as a thin wrapper around Terraform, offering additional features to streamline and optimise Terraform usage. Key functions of Terragrunt include helping users keep their Terraform configurations DRY (Don’t Repeat Yourself), efficiently managing multiple Terraform modules, and handling remote state management. By reducing repetition in Terraform code and simplifying the management of complex module dependencies and remote state, Terragrunt makes working with Terraform more efficient, especially for larger or more complex infrastructure deployments.

locals { # Automatically load region-level variables region_vars = read_terragrunt_config(find_in_parent_folders("region.hcl")) # Automatically load environment-level variables` environment_vars = read_terragrunt_config(find_in_parent_folders("env.hcl")) # Extract the variables we need for easy access account_name = local.environment_vars.locals.account_name account_id = local.environment_vars.locals.aws_account_id aws_region = local.region_vars.locals.aws_region # This is the S3 bucket where the Terraform State Files will be stored remote_state_bucket = "devops-bucket" # This is the DynamoDB table where Terraform will add the locking status dynamodb_table = "terraform-state-lock" # IAM Role for Terraform backend to assume terraform_backend_role = "arn:aws:iam::{shared-services_account_id}:role/terraform-backend-role" environment_path = replace(path_relative_to_include(), "environments/", "") # https://github.com/hashicorp/terraform/releases terraform_version = "latest" # https://github.com/gruntwork-io/terragrunt/releases terragrunt_version = "latest" } # Generate an AWS provider block generate "provider" { path = "provider.tf" if_exists = "overwrite_terragrunt" contents = <

If your team works with Terraform, you should definitely try Terragrunt (and obviously, its CLI tool!). It's an Infrastructure as Code (IaC) tool that acts as a wrapper for Terraform and simplifies dealing with multiple Terraform modules in different environments.

I’ve been using Terragrunt [0] for the past three years to manage loosely coupled stacks of Terraform configurations. It allows you to compose separate configurations almost as easily as you compose modules within a configuration. Its got its own learning curve, but its a solid tool to have in the tool box.

[0] https://terragrunt.gruntwork.io/

Documentation is really important, so having a README.md file inside your Terraform repository that explains how to use the automation (including descriptions of variables and outputs) really help in understanding what has been implemented. To easily generate the description of variables and outputs, you can leverage tfdocs.

‍Terraform-docs is a tool that automatically generates documentation from Terraform modules in various output formats, including markdown, JSON, and others. It's particularly useful for maintaining up-to-date documentation of your Terraform modules' inputs, outputs, providers, and resources.

Earlier this year I wrote about the challenges I faced creating a Terraform module. I mentioned then that I was leveraging terraform-docs and GitHub Actions to automate documentation, but a full workflow walkthrough was out of that post's scope.

### Muti-Stage pipeline for windows trigger: - main variables: terraformDocsVersion: '0.16.0' serviceConnectionName: 'Terraform-SPN-DevOps-MagiconionM' keyvaultName: 'pwd9000-core-kv' pool: vmImage: 'windows-latest' stages: - stage: GenerateTerraformDocumentation jobs: - job: Generate_Terraform_Documentation steps: - checkout: self persistCredentials: true # this allows the later scripts to use the system-provided git token to push changes back to the repo ### Link to key vault. - task: AzureKeyVault@1 inputs: azureSubscription: $(serviceConnectionName) #ADO service connection (Service principal) KeyVaultName: $(keyvaultName) SecretsFilter: 'TerraformDocsPAToken' RunAsPreJob: true displayName: Get PAToken from Keyvault ### Install Terraform-Docs. - powershell: | Invoke-WebRequest -Uri "https://github.com/terraform-docs/terraform-docs/releases/download/v$(terraformDocsVersion)/terraform-docs-v$(terraformDocsVersion)-windows-amd64.zip" -OutFile "terraform-docs.zip" Expand-Archive -Path "terraform-docs.zip" -DestinationPath "$(System.DefaultWorkingDirectory)\terraform-docs" -Force $env:Path += ";$(System.DefaultWorkingDirectory)\terraform-docs" displayName: 'Install terraform-docs' ### Remove all old README.md files and generate new README.md files for each TF module. - powershell: | # Set Modules Root Directory Set-Location "$(Build.SourcesDirectory)\@TF_Modules" # Get all subdirectories (Terraform module directories) $terraformModuleDirs = Get-ChildItem -Path (Get-Location) -Directory # Loop through each directory to cleanup/remove old README files foreach ($dir in $terraformModuleDirs) { # Get all files in the directory $readMeFiles = Get-ChildItem -Path $dir.FullName -Filter 'README.md' # Loop through each file in each terraform module foreach ($file in $readMeFiles) { # Check if README file already exists and remove if ($file) { # Remove the file Remove-Item $file.FullName -Confirm:$false Write-Output "Old file '$($file.Name)' removed from '$($dir.FullName)'" } } #After cleanup create a new README.md file with 'terraform-docs' based on latest TF module code in current folder(terraform module) $tfFiles = Get-ChildItem -Path $dir.FullName -Filter *.tf if ($tfFiles.Count -gt 0) { # Create new README.md file $(System.DefaultWorkingDirectory)\terraform-docs\terraform-docs.exe markdown table $dir.FullName --output-file "README.md" } else { Write-Output "No .tf files found." } } displayName: 'Cleanup and Generate new README for each TF module' ### Commit and push updated README.md files for TF modules. - powershell: | git config --local user.email "[email protected]" git config --local user.name "Terraform Docs" git add *.md git commit -m "Update README.md for each TF module" git push origin HEAD:$(Build.SourceBranchName) displayName: 'Commit and Push updated README.md files for TF modules' env: SYSTEM_ACCESSTOKEN: $(TerraformDocsPAToken)

I'm going to have to figure out how to run terraform-docs.io against each module. I have zero terraform experience.

Terraform-docs: This tool generates documentation for your Terraform modules in various formats, such as Markdown, HTML, and JSON. Terraform-docs parses your Terraform code and extracts documentation from comments, variable and output descriptions, and input/output examples. The tool then generates a table of contents with links to the relevant documentation for each module, which makes it easy to understand the purpose and usage of each module.

Terraform docs

I have been using https://terraform-docs.io to generate documentation of my terraform modules. However creating nested objects with optionals makes the documentation rather ugly and hard to read. See gh issue which also describes this type of issue: https://github.com/terraform-docs/terraform-docs/issues/656

I am looking for a way to automate doc creation and updates for ansible roles. I recently discovered terraform-docs and I am curious if there is something similar for ansible.

terraform-docs: https://github.com/terraform-docs/terraform-docs

Do you have other Terraform-related open-source tools that you recommend? Please go ahead and add in the comments!