mazzle-starter
terragrunt
mazzle-starter | terragrunt | |
---|---|---|
3 | 56 | |
9 | 7,621 | |
- | 0.9% | |
3.4 | 9.2 | |
7 months ago | 6 days ago | |
HCL | Go | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
mazzle-starter
-
Scaling Relational SQL Databases
I take advantage of I/O being parallel in Python in my mazzle continuous integration pipeline tool. I'm not sharing mutable state.
I spin up a graph of python Threads and each joins others in a graph. This way we can run graphs in parallel. See this graph - the parts that look like this:
dependency -> {parallel1; parallel2; parallel3} -> postparallel
parallel1, parallel2, parallel3 can run in parallel in a separate python thread because the IO is parallel.
postparallel joins parallel1, parallel2, parallel3 and waits for them all to complete.
Where parallel1-3 is things like ansible, packer (slow), AMI builds, chef runs etc.
https://github.com/samsquire/mazzle-starter/blob/master/arch...
- Terraform v1.0 Is Out
-
Terraform 1.0 Release
I use a tool I wrote to layer my infrastructure with layers called components and I configure it with a Graphviz file.
My tool, called mazzle (previously devops-pipeline) would run parts of the graph that can run in parallel in parallel. It can also run parts of the build on SSH workers. You bring up the workers at the beginning of the build.
Here's an example of a graph generated from graphviz file: https://github.com/samsquire/mazzle/blob/master/docs/archite...
This graph brings up a hashicorp vault server, Java application, bastion proxy etc.
here's the graphviz file: https://github.com/samsquire/mazzle-starter/blob/master/arch...
It describes the ordering of the infrastructure, the invocation of Ansible, packer, shell scripts to set up vault etc.
The idea is to be able to bring up a new environment by changing one parameter. There's a React GUI too.
https://devops-pipeline.com
terragrunt
-
Deploying a Containerized App to ECS Fargate Using a Private ECR Repo & Terragrunt
name: Configure on: push: branches: - main pull_request: branches: - main workflow_dispatch: inputs: destroy: description: 'Run Terragrunt destroy command' required: true default: 'false' type: choice options: - true - false jobs: apply: if: ${{ !inputs.destroy || inputs.destroy == 'false' }} runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v2 - name: Setup SSH uses: webfactory/[email protected] with: ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }} - name: Setup Terraform uses: hashicorp/setup-terraform@v2 with: terraform_version: 1.5.5 terraform_wrapper: false - name: Setup Terragrunt run: | curl -LO "https://github.com/gruntwork-io/terragrunt/releases/download/v0.48.1/terragrunt_linux_amd64" chmod +x terragrunt_linux_amd64 sudo mv terragrunt_linux_amd64 /usr/local/bin/terragrunt terragrunt -v - name: Apply Terraform changes run: | cd dev terragrunt run-all apply -auto-approve --terragrunt-non-interactive -var AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -var AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -var AWS_REGION=$AWS_DEFAULT_REGION env: AWS_ACCESS_KEY_ID: ${{ vars.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} AWS_DEFAULT_REGION: ${{ vars.AWS_DEFAULT_REGION }} destroy: if: ${{ inputs.destroy == 'true' }} runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v2 - name: Setup SSH uses: webfactory/[email protected] with: ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }} - name: Setup Terraform uses: hashicorp/setup-terraform@v2 with: terraform_version: 1.5.5 terraform_wrapper: false - name: Setup Terragrunt run: | curl -LO "https://github.com/gruntwork-io/terragrunt/releases/download/v0.48.1/terragrunt_linux_amd64" chmod +x terragrunt_linux_amd64 sudo mv terragrunt_linux_amd64 /usr/local/bin/terragrunt terragrunt -v - name: Destroy Terraform changes run: | cd dev terragrunt run-all destroy -auto-approve --terragrunt-non-interactive -var AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -var AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -var AWS_REGION=$AWS_DEFAULT_REGION env: AWS_ACCESS_KEY_ID: ${{ vars.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} AWS_DEFAULT_REGION: ${{ vars.AWS_DEFAULT_REGION }}
-
Top Terraform Tools to Know in 2024
Terragrunt is a thin wrapper that provides extra tools for keeping your Terraform configurations DRY (Don't Repeat Yourself), working with multiple Terraform modules, and managing remote state. It's particularly useful in managing large-scale infrastructure deployments with Terraform.
-
DevSecOps with AWS- IaC at scale - Building your own platform - Part 1
... #************************** Terraform ************************************* ARG TERRAFORM_VERSION=1.7.3 RUN set -ex \ && curl -O https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip && unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d /usr/local/bin/ RUN set -ex \ && mkdir -p $HOME/.terraform.d/plugin-cache && echo 'plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"' > ~/.terraformrc #************************* Terragrunt ************************************* ARG TERRAGRUNT_VERSION=0.55.1 RUN set -ex \ && wget https://github.com/gruntwork-io/terragrunt/releases/download/v${TERRAGRUNT_VERSION}/terragrunt_linux_amd64 -q \ && mv terragrunt_linux_amd64 /usr/local/bin/terragrunt \ && chmod +x /usr/local/bin/terragrunt #*********************** Terramate **************************************** ARG TERRAMATE_VERSION=0.4.5 RUN set -ex \ && wget https://github.com/mineiros-io/terramate/releases/download/v${TERRAMATE_VERSION}/terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && tar -xzf terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && mv terramate /usr/local/bin/terramate \ && chmod +x /usr/local/bin/terramate #*********************** tfsec ******************************************** ARG TFSEC_VERSION=1.28.5 RUN set -ex \ && wget https://github.com/aquasecurity/tfsec/releases/download/v${TFSEC_VERSION}/tfsec-linux-amd64 \ && mv tfsec-linux-amd64 /usr/local/bin/tfsec \ && chmod +x /usr/local/bin/tfsec \ && terragrunt --version #**********************Terraform docs ************************************ ARG TERRRAFORM_DOCS_VERSION=0.17.0 RUN set -ex \ && curl -sSLo ./terraform-docs.tar.gz https://terraform-docs.io/dl/v${TERRRAFORM_DOCS_VERSION}/terraform-docs-v${TERRRAFORM_DOCS_VERSION}-$(uname)-amd64.tar.gz \ && tar -xzf terraform-docs.tar.gz \ && chmod +x terraform-docs \ && mv terraform-docs /usr/local/bin/terraform-docs #********************* ShellCheck ***************************************** ARG SHELLCHECK_VERSION="stable" RUN set -ex \ && wget -qO- "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION?}/shellcheck-${SHELLCHECK_VERSION?}.linux.x86_64.tar.xz" | tar -xJv \ && cp "shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/bin/ \ && shellcheck --version ...
-
Self-service infrastructure as code
Our first attempt was to introduce other engineering teams to Terraform - the Platform team was already using it extensively with Terragrunt, and using Atlantis to automate plan and apply operations in a Git flow to ensure infrastructure was consistent. We'd written modules, with documentation, and an engineer would simply need to raise a PR to use the module and provide the right values, and Atlantis (once the PR was approved by Platform) would go ahead and set it up for them.
-
Shielding Your Apps in the Cloud: Integrating CloudFront and AWS WAF with Terraform
Terragrunt: An extension of Terraform, Terragrunt assists in managing complex infrastructure with less duplication and more efficiency. Its power lies in its ability to manage dependencies and its dry configuration approach.
-
Advanced Terraform: Getting Started With Terragrunt
Copy the link and download on your terminal using the wget command. Example: wget https://github.com/gruntwork-io/terragrunt/releases/download/v0.54.19/terragrunt_linux_amd64
-
EC2 Configuration using Ansible & GitHub Actions
name: Configure on: push: branches: - main pull_request: branches: - main jobs: terraform: runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v2 - name: Setup SSH uses: webfactory/[email protected] with: ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }} - name: Setup Terraform uses: hashicorp/setup-terraform@v2 with: terraform_version: 1.5.5 terraform_wrapper: false - name: Setup Terragrunt run: | curl -LO "https://github.com/gruntwork-io/terragrunt/releases/download/v0.48.1/terragrunt_linux_amd64" chmod +x terragrunt_linux_amd64 sudo mv terragrunt_linux_amd64 /usr/local/bin/terragrunt terragrunt -v - name: Apply Terraform changes run: | cd dev terragrunt run-all apply -auto-approve --terragrunt-non-interactive -var AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -var AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -var AWS_REGION=$AWS_DEFAULT_REGION cd apache-server/ec2-web-server public_ip=$(terragrunt output instance_public_ip) echo "$public_ip" > public_ip.txt cat public_ip.txt env: AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }} - name: Upload artifact uses: actions/upload-artifact@v4 with: name: ip-artifact path: dev/apache-server/ec2-web-server/public_ip.txt ansible: runs-on: ubuntu-latest needs: terraform steps: - name: Download artifact uses: actions/download-artifact@v4 with: name: ip-artifact - name: Configure Ansible run: | sudo apt update sudo pipx inject ansible-core jmespath ansible-playbook --version sudo echo "[web]" >> ansible_hosts sudo cat public_ip.txt >> ansible_hosts mv ansible_hosts $HOME sudo cat $HOME/ansible_hosts - name: Configure playbook run: | cd $HOME cat > deploy.yml < Test Page This is a test page EOF cat $HOME/deploy.yml - name: Run playbook uses: dawidd6/action-ansible-playbook@v2 with: playbook: deploy.yml directory: /home/runner key: ${{secrets.SSH_PRIVATE_KEY}} options: | --inventory ansible_hosts --verbose
-
Top 10 terraform tools you should know about.
Created and maintained by Gruntwork, Terragrunt is a tool designed to enhance Terraform’s capabilities. It acts as a thin wrapper around Terraform, offering additional features to streamline and optimise Terraform usage. Key functions of Terragrunt include helping users keep their Terraform configurations DRY (Don’t Repeat Yourself), efficiently managing multiple Terraform modules, and handling remote state management. By reducing repetition in Terraform code and simplifying the management of complex module dependencies and remote state, Terragrunt makes working with Terraform more efficient, especially for larger or more complex infrastructure deployments.
-
Seamless Cloud Infrastructure: Integrating Terragrunt and Terraform with AWS
locals { # Automatically load region-level variables region_vars = read_terragrunt_config(find_in_parent_folders("region.hcl")) # Automatically load environment-level variables` environment_vars = read_terragrunt_config(find_in_parent_folders("env.hcl")) # Extract the variables we need for easy access account_name = local.environment_vars.locals.account_name account_id = local.environment_vars.locals.aws_account_id aws_region = local.region_vars.locals.aws_region # This is the S3 bucket where the Terraform State Files will be stored remote_state_bucket = "devops-bucket" # This is the DynamoDB table where Terraform will add the locking status dynamodb_table = "terraform-state-lock" # IAM Role for Terraform backend to assume terraform_backend_role = "arn:aws:iam::{shared-services_account_id}:role/terraform-backend-role" environment_path = replace(path_relative_to_include(), "environments/", "") # https://github.com/hashicorp/terraform/releases terraform_version = "latest" # https://github.com/gruntwork-io/terragrunt/releases terragrunt_version = "latest" } # Generate an AWS provider block generate "provider" { path = "provider.tf" if_exists = "overwrite_terragrunt" contents = <
-
Top 10 CLI Tools for DevOps Teams
If your team works with Terraform, you should definitely try Terragrunt (and obviously, its CLI tool!). It's an Infrastructure as Code (IaC) tool that acts as a wrapper for Terraform and simplifies dealing with multiple Terraform modules in different environments.
What are some alternatives?
dhall-terraform - Generate dhall records from terraform resouces, data_sources & providers
terraform-cdk - Define infrastructure resources using programming constructs and provision them using HashiCorp Terraform
terraform - Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
aws-cloudformation-res
Pulumi - Pulumi - Infrastructure as Code in any programming language. Build infrastructure intuitively on any cloud using familiar languages 🚀
terraform-ls - Terraform Language Server
LocalStack - 💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline
pulumi-tf-provider-boilerplate - Boilerplate code for Terraform provider-backed Pulumi packages
atlantis - Terraform Pull Request Automation
porter - Porter enables you to package your application artifact, client tools, configuration and deployment logic together as an installer that you can distribute, and install with a single command.
sops - Simple and flexible tool for managing secrets