aws-nuke VS vscode-infracost

To give this a slightly different spin:

--> "The best optimization is simply not spinning things up."

At least for local development and testing, as made possible by LocalStack (https://localstack.cloud), among other local testing solutions and emulators.

We've seen so many teams fall into the trap of "someone forgot to shut down dev resource X for a week and now we've racked up a $$$ bill on AWS".

What is everyone's strategy to avoid this kind of situation? Tools like `aws-nuke` (https://github.com/rebuy-de/aws-nuke) are awesome (!) to clean up unused resources, but frankly they should not be necessary in the first place.

We use nuke aws at work to remove any leftovers: https://github.com/rebuy-de/aws-nuke

You can use this tool on github to nuke all resources.

There's this. I haven't used it myself, but it looks to be pretty effective: https://github.com/rebuy-de/aws-nuke

At this scale, operations can take a lot of time, because there are multiple operational tasks that we need to do when AWS accounts are leaving the AWS Organization or Teams are nuking the AWS account, StackSets Instances get drifted, because not all required resources for compliance can be secured ( SCP Limitations ), existing AWS accounts are joining the AWS Organization and all mandatory StackSets needs to be deployed, and manual steps should be reduced to a minimum. Furthermore, there is no feature from the Service itself to gain an overview of the status of drifted Instances and the general health of your StackSet health and compliance.

Since you're using CDK already, have a way to configure the deployment of the whole thing to a per-developer test account; that's still gonna cost you, but you can bundle everything in an organization / organizational unit for billing purposes, and you can also schedule https://github.com/rebuy-de/aws-nuke to run nightly to clean these accounts from longer-running resources.

And by extension, https://github.com/rebuy-de/aws-nuke as well.

I can also recommend aws-nuke which is an easy to way to destroy in your account.

nothing blew up accidentally this week, but our team at kubefirst is falling more and more in love with aws-nuke. it's an open source command line tool that lets you basically reset an aws account back to an empty state. if you have an environment where you regularly practice your platform provisioning, you probably know that failed destroys while iterating on orchestration can leave junk behind pretty easily. aws-nuke has been so nice to be able to blow away everything in an aws account - and then we just run terraform in the account to get all our core infra back afterward. nice allowlist filters and dryrun detail work too. check them out.

Hi, we are Ali, Hassan, and Alistair, co-founders of Infracost (https://www.infracost.io/). Infracost helps engineers see the cost of each Terraform change before launching resources. When changes are made, it posts a comment with the cloud cost impact. For example, “you’ve added 2 instances and volumes, and change an instance type from medium to large, your bill will increase by 25% next month, from $1000 to $1250 per month”.

We launched in February 2021 (https://news.ycombinator.com/item?id=26064588), and Infracost is now being actively used by over 3,000 companies. However, there is a shift happening in the cloud cost management space. New teams, called FinOps teams (a combination of "Finance" and "DevOps"), are being formed within companies to manage cloud costs.

One of the first tasks assigned to these teams is to determine "who is using what" - that is, which teams, business units, products, etc. are spending the most on cloud. To accomplish this, they use tags. Tags are labels that all cloud resources should have and are key-value pairs. For example, a server could be tagged with: product=HackerNews; environment=production; team=blueTeam. So if resources are not tagged properly, then you can’t tell who is using what.

However, FinOps teams face challenges because their tools are reactive. These tools begin by analyzing cloud bills and providing visibility of tags from there. This means that they are looking at resources that are already running in production and costing money. A customer recently shared, “I want all resources to be properly tagged. But if they are not, I would rather a resource not be tagged at all than be tagged incorrectly.”

My "aha" moment! FinOps teams can define a tagging policy that can be validated in CI/CD before resources are launched. This is important because if code is shipped with the wrong tags, FinOps teams will have to fight for sprint time to fix them. Even if you shut down an untagged resource directly in the cloud, the next time Terraform runs, the resource will launch again with no tag. You need to fix the issue at its root.

I’d love your feedback on our solution to the tagging problem. You define your tag key-value policy in our SaaS product, and Infracost checks all Terraform resources per change. If anything fails the policy, it posts a comment with the details of which resources need tags, and what the allowed values are. Once fixed, it will let the code be shipped to production.

Try it out by going to https://dashboard.infracost.io/, setting up with the GitHub app or GitLab app, and defining your tagging policy. It will then scan your repository and inform you of any missing tags and their file and line number. You can use the free trial, but if you need more time, please message me and I’ll extend it for you.

I would also love to hear how others ensure that the correct tag keys and values are applied to all resources, and whether this is done proactively or reactively. Additionally, I would be interested in hearing about any lessons learned in the process.

Cheers

Great toolchain, including Infracost or tfsec.

Infracost is an open-source tool for estimating the cost of cloud infrastructure using Infrastructure as Code templates.

Throughout this post, you will see me mention the cost of running this using an estimate. I have been using for a while, a tool called infracost which is an open source (with subscription based additions) cost estimator tool - https://www.infracost.io/. For this demonstration, using the sample code listed above, it would cost an estimated $76.65/month - so if you don't want rack up a bill, only deploy when you want to test, and use Terraform to destroy the services when you are done.

Infracost : Infracost is an open-source tool that allows users to see the cost of running their infrastructure, such as AWS resources, in near real-time. It uses the AWS Price List API to determine the costs of resources, and can be integrated into CI/CD pipelines to provide cost feedback during the development process. This allows developers to make informed decisions about their infrastructure and optimize costs. Additionally, Infracost can be used to create alerts based on cost thresholds, so you can be notified when your infrastructure costs exceed a certain amount. This can be especially useful for teams that operate on a tight budget or need to manage costs closely.

An Infracost API key. You can get one by signing up for free at Infracost.io.

Since Wing compiles to Terraform, you should be able to manually run Infracost (https://www.infracost.io) to get an idea of costs.

Having that capability baked into the language/compiler would be a great addition.