Cutting down AWS cost by $150k per year simply by shutting things off

• cloud-custodian

  32 5,201 9.5 Python

  Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources

  

> The best optimization is simply shutting things off

This is the way.

A similar idea has been bouncing around in my mind for a while now. An ideal, turnkey system would do the following:

- Execute via Lambda (serverless).

- Support automated startup and shutdown of various AWS resources on a schedule influenced by specially formatted tags.

- Enable resources to be brought back up out of schedule when demand dictates.

- Operate as a TCP/HTTP proxy that can delay clients so that a given service can be started when it is dormant or, even better, the service isn't serverless but you want it to be. This can't work for everything, but perhaps enough things such that the need to run always on services is reduced.

Cloud Custodian [1] can purportedly do some of this, but I've been reluctant to learn yet another YAML-based DSL to use it.

So this is my "make things designed to be always-on serverless instead" project and the work AWS has done to make Java apps function on Lambda keeps me thinking about the potential to take things that 1) have a relatively long startup time and 2) are designed to be long running service loops, and find a way to force them into the serverless execution model.

[1] https://cloudcustodian.io/

• steampipe

  146 6,391 9.7 Go

  Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.

  

Readers may find Steampipe's [1] AWS Thrifty Mod [2] useful. It will automatically scan multiple accounts and regions for 50 cost saving opportunities - many of which are looking for over-provisioned or unused resources. For example, it's crazy how much you can save by doing things like just converting your EBS volumes to the newer gp3 type. Combine with Flowpipe [3] to automate checks and actions. It's all open source and extensible.

1 - https://github.com/turbot/steampipe

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• steampipe-mod-aws-thrifty

  4 93 6.6 HCL

  Are you a Thrifty AWS dev? This mod checks your AWS accounts for unused and under-utilized resources using Powerpipe and Steampipe.

  

2 - https://github.com/turbot/steampipe-mod-aws-thrifty

• flowpipe

  3 276 9.9 Go

  Flowpipe is a cloud scripting engine. Automation and workflow to connect your clouds to the people, systems and data that matters.

  

• vscode-infracost

  27 1,785 6.1 TypeScript

  See cost estimates for Terraform right in your editor💰📉

  

• aws-nuke

  86 5,338 8.0 Go

  Nuke a whole AWS account and delete all its resources.

  

To give this a slightly different spin:

--> "The best optimization is simply not spinning things up."

At least for local development and testing, as made possible by LocalStack (https://localstack.cloud), among other local testing solutions and emulators.

We've seen so many teams fall into the trap of "someone forgot to shut down dev resource X for a week and now we've racked up a $$$ bill on AWS".

What is everyone's strategy to avoid this kind of situation? Tools like `aws-nuke` (https://github.com/rebuy-de/aws-nuke) are awesome (!) to clean up unused resources, but frankly they should not be necessary in the first place.

• LocalStack

  154 52,145 9.9 Python

  💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline

  

To give this a slightly different spin:

--> "The best optimization is simply not spinning things up."

At least for local development and testing, as made possible by LocalStack (https://localstack.cloud), among other local testing solutions and emulators.

We've seen so many teams fall into the trap of "someone forgot to shut down dev resource X for a week and now we've racked up a $$$ bill on AWS".

What is everyone's strategy to avoid this kind of situation? Tools like `aws-nuke` (https://github.com/rebuy-de/aws-nuke) are awesome (!) to clean up unused resources, but frankly they should not be necessary in the first place.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project