awsrm VS steampipe

It's because I build the tooling in way that Terraform resource names, such as aws_vpc, aws_instance, aws_iam_role, etc. can be used to find resources (with awsls) and delete them (with awsrm), instead of using the to me not so familiar and inconsistent AWS CLI (or CloudFormation) resource names with commands, such as aws ec2 describe-vpcs or aws ec2 delete-vpc (in some cases it's list instead of describe, e.g., for roles it's aws iam list-roles, which is hard to remember).

Readers may also enjoy Steampipe [1], an open source tool to live query 140+ services with SQL (e.g. AWS, GitHub, CSV, Kubernetes, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc with other tables. (Disclaimer - I'm a lead on the project.)

1 - https://github.com/turbot/steampipe

I really really like Steampipe to do this kind of query: https://steampipe.io, which is essentially PostgreSQL (literally) to query many different kind of APIs, which means you have access to all PostgreSQL's SQL language can offer to request data.

They have a Kubernetes plugin at https://hub.steampipe.io/plugins/turbot/kubernetes and there are a couple of things I really like:

* it's super easy to request multiple Kubernetes clusters transparently: define one Steampipe "connection" for each of your clusters + define an "aggregator" connection that aggregates all of them, then query the "aggregator" connection. You will get a "context" column that indicates which Kubernetes cluster the row came from.

I was just trying to get a sense for where the line of demarcation was in your mind. PG has "foreign data wrappers"[1] that allow one to treat external ... whatever ... as if it were a table or procedure within PG. Just stupid powerful, IMHO. It is FDW-specific whether "transaction" means anything to the foreign system, so that may break your mental model but could still get you very close (e.g. BEGIN; UPDATE s3 SET ...; ROLLBACK; may not do anything sensible)

https://github.com/turbot/steampipe#steampipe-plugins and https://steampipe.io/docs/steampipe_postgres/overview may be relevant, although watch out for Steampipe's license

https://github.com/topics/foreign-data-wrapper and https://github.com/topics/fdw are some other examples

1: https://www.postgresql.org/docs/17/fdwhandler.html (although strictly speaking that page is for _authoring_ FDW, not a tl;dr of the concept)

https://steampipe.io/ https://github.com/prowler-cloud/prowler https://former2.com

https://steampipe.io/ showcases some really interesting scenarios for using FDWs in place of regular ETL and API integrations

Steampipe is a tool for querying cloud APIs and other data sources using SQL in a zero-ETL manner.

Readers may also enjoy Steampipe [1], an open source CLI to live query Google Sheets [2] and 140+ other services with SQL (e.g. AWS, GitHub, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc across the services. (Disclaimer - I'm a lead on the project.)

1 - https://github.com/turbot/steampipe

be mindful of its AGPLv3 https://github.com/turbot/steampipe/blob/v0.21.8/LICENSE (AFAIK v0.4.3 is the last MIT release https://github.com/turbot/steampipe/blob/v0.4.3/LICENSE ) and the actual providers are Apache 2 <https://github.com/turbot/steampipe-plugin-aws/blob/v0.131.0...> (but I don't know if provider drift makes them compatible with 0.4 or not)

iasql seems to be AWS only, but good for them for taking this on: