cloud-custodian VS jmespath.py

> The best optimization is simply shutting things off

This is the way.

A similar idea has been bouncing around in my mind for a while now. An ideal, turnkey system would do the following:

- Execute via Lambda (serverless).

- Support automated startup and shutdown of various AWS resources on a schedule influenced by specially formatted tags.

- Enable resources to be brought back up out of schedule when demand dictates.

- Operate as a TCP/HTTP proxy that can delay clients so that a given service can be started when it is dormant or, even better, the service isn't serverless but you want it to be. This can't work for everything, but perhaps enough things such that the need to run always on services is reduced.

Cloud Custodian [1] can purportedly do some of this, but I've been reluctant to learn yet another YAML-based DSL to use it.

So this is my "make things designed to be always-on serverless instead" project and the work AWS has done to make Java apps function on Lambda keeps me thinking about the potential to take things that 1) have a relatively long startup time and 2) are designed to be long running service loops, and find a way to force them into the serverless execution model.

[1] https://cloudcustodian.io/

Goal was to clear up anything old and set us up to rotate keys in use. Why did I do it in the end of December? It was a quarterly goal and I learned to push those across the line if I wanted a good review. Great incentive, that one. I used Cloud Custodian for this. It has a terrible bug where the code says you'll be acting on days since the key was used but actually is reading days since it was created.

What is wrong with https://github.com/cloud-custodian/cloud-custodian?

You can start reading about it here.

Use a tool like this https://cloudcustodian.io/ to manage instance on/off hours or go fargate.

Certs aside, there are some great compliance tools out there that you might want to become familiar with. Here is one that comes to mind (is open-source): https://cloudcustodian.io/

I'd use a combination of Cloudcustodian for start/stop scheduling and Apprise for notifications.

Cloud custodian is a good utility if config rules doesn’t satisfy your needs. It’s also cross platform.

AWS Security with open source – Cloud Custodian is a Cloud Security Posture Management (CSPM) tool. CSPM tools evaluate your cloud configuration and identify common configuration mistakes. They also monitor cloud logs to detect threats and configuration changes.

The Azure CLI lets us write queries to filter the results of the az storage blob list command. The queries are written in JMESPath, which is a query language for JSON. In this case, we are filtering the results to only include blobs that end with the .bacpac extension and then selecting the first one as ordered by the lastModified property. If there are no blobs found, the script exits with a failure code. If we find a blob, we download it to the local path specified by the localPath variable.

For JSON there is the `jmespath` library which might help.

https://github.com/jmespath/jmespath.py

I love jq, but I also use JMESPath (especially with AWS CLI), yq (bundled with tomlq and xq as well), and dasel [2]. I also wish hclq [3] wasn't so dead!

[0]: https://jmespath.org/

Probably writing the query side of things is a lot of the fun here, but there is actually a spec (and a complying Rust impl) you can hook into for this JQ-like querying: https://jmespath.org/ ( https://github.com/jmespath/jmespath.rs ).

Thanks to custom variables that use labels, we can create various filters for the dashboard. You can look up my configuration of variables and extend it with an analogy way for your own needs. At the top, I marked the filter with detected pods in selected namespace. In the lower part, you can see a preview of all labels that are associated with a single log line. Most labels are meta information that Promtail adds during scraping targets. This part of the Promtail configuration provides it. In this section, I also marked a few labels that not comes out-of-the box e.g. leavel , class , thread . We added these labels using the Promtail json stage. You need to know that Promtail processes scraped logs in a pipeline. A pipeline is comprised of a set of stages. json stage is a parsing stage that reads the log line as JSON and accepts JMESPath expressions to extract data.

The tester on the official website for JMESPath (what json_query is doing) has been useful to me: https://jmespath.org/

Interesting. How does this compared to Jmespath? Not saying Jmespath is superior, just wondering whether you were aware of it.