aws-nuke VS cloud-custodian

To give this a slightly different spin:

--> "The best optimization is simply not spinning things up."

At least for local development and testing, as made possible by LocalStack (https://localstack.cloud), among other local testing solutions and emulators.

We've seen so many teams fall into the trap of "someone forgot to shut down dev resource X for a week and now we've racked up a $$$ bill on AWS".

What is everyone's strategy to avoid this kind of situation? Tools like `aws-nuke` (https://github.com/rebuy-de/aws-nuke) are awesome (!) to clean up unused resources, but frankly they should not be necessary in the first place.

We use nuke aws at work to remove any leftovers: https://github.com/rebuy-de/aws-nuke

You can use this tool on github to nuke all resources.

There's this. I haven't used it myself, but it looks to be pretty effective: https://github.com/rebuy-de/aws-nuke

At this scale, operations can take a lot of time, because there are multiple operational tasks that we need to do when AWS accounts are leaving the AWS Organization or Teams are nuking the AWS account, StackSets Instances get drifted, because not all required resources for compliance can be secured ( SCP Limitations ), existing AWS accounts are joining the AWS Organization and all mandatory StackSets needs to be deployed, and manual steps should be reduced to a minimum. Furthermore, there is no feature from the Service itself to gain an overview of the status of drifted Instances and the general health of your StackSet health and compliance.

Since you're using CDK already, have a way to configure the deployment of the whole thing to a per-developer test account; that's still gonna cost you, but you can bundle everything in an organization / organizational unit for billing purposes, and you can also schedule https://github.com/rebuy-de/aws-nuke to run nightly to clean these accounts from longer-running resources.

And by extension, https://github.com/rebuy-de/aws-nuke as well.

I can also recommend aws-nuke which is an easy to way to destroy in your account.

nothing blew up accidentally this week, but our team at kubefirst is falling more and more in love with aws-nuke. it's an open source command line tool that lets you basically reset an aws account back to an empty state. if you have an environment where you regularly practice your platform provisioning, you probably know that failed destroys while iterating on orchestration can leave junk behind pretty easily. aws-nuke has been so nice to be able to blow away everything in an aws account - and then we just run terraform in the account to get all our core infra back afterward. nice allowlist filters and dryrun detail work too. check them out.

> The best optimization is simply shutting things off

This is the way.

A similar idea has been bouncing around in my mind for a while now. An ideal, turnkey system would do the following:

- Execute via Lambda (serverless).

- Support automated startup and shutdown of various AWS resources on a schedule influenced by specially formatted tags.

- Enable resources to be brought back up out of schedule when demand dictates.

- Operate as a TCP/HTTP proxy that can delay clients so that a given service can be started when it is dormant or, even better, the service isn't serverless but you want it to be. This can't work for everything, but perhaps enough things such that the need to run always on services is reduced.

Cloud Custodian [1] can purportedly do some of this, but I've been reluctant to learn yet another YAML-based DSL to use it.

So this is my "make things designed to be always-on serverless instead" project and the work AWS has done to make Java apps function on Lambda keeps me thinking about the potential to take things that 1) have a relatively long startup time and 2) are designed to be long running service loops, and find a way to force them into the serverless execution model.

[1] https://cloudcustodian.io/

Goal was to clear up anything old and set us up to rotate keys in use. Why did I do it in the end of December? It was a quarterly goal and I learned to push those across the line if I wanted a good review. Great incentive, that one. I used Cloud Custodian for this. It has a terrible bug where the code says you'll be acting on days since the key was used but actually is reading days since it was created.

What is wrong with https://github.com/cloud-custodian/cloud-custodian?

You can start reading about it here.

Use a tool like this https://cloudcustodian.io/ to manage instance on/off hours or go fargate.

Certs aside, there are some great compliance tools out there that you might want to become familiar with. Here is one that comes to mind (is open-source): https://cloudcustodian.io/

I'd use a combination of Cloudcustodian for start/stop scheduling and Apprise for notifications.

Cloud custodian is a good utility if config rules doesn’t satisfy your needs. It’s also cross platform.

AWS Security with open source – Cloud Custodian is a Cloud Security Posture Management (CSPM) tool. CSPM tools evaluate your cloud configuration and identify common configuration mistakes. They also monitor cloud logs to detect threats and configuration changes.