cloudquery VS steampipe-mod-aws-compliance

Check out CloudQuery - https://github.com/cloudquery/cloudquery for an easy cloud asset inventory.

There both do something like what you're looking for.... https://github.com/cloudquery/cloudquery https://github.com/openraven/magpie

Unified API is a holly grail but as many said quite difficult to abstract every use case in a scalable way that won't break. At CloudQuery (https://github.com/cloudquery/cloudquery) we focus solely on the ELT use-case(Founder/Maintainer here).

Congrats!! How does it compare to the ELT space and the modern data stack where you have ingestion/storage/visualization layers decoupled?

Asking as the founder of CloudQuery (https://github.com/cloudquery/cloudquery), Saw Datasette quite a few times around data exploration but curious to hear about the most popular use-cases of Datasette!

Congrats!! We also focus on performance at CloudQuery (https://github.com/cloudquery/cloudquery) by using Golang, gRPC and still trying to be abstract enough to support different databases :)

In any case good luck!

CloudQuery for ETL

Another ELT framework that's an alternative to Airbyte

Another alternate ELT

Check out CloudQuery, we have PostgreSQL source connectors and S3 destination that supports parquet (Disclaimer: Maintainer and founder here)

Hello! Im Yevgeny, Founder & maintainer at CloudQuery . We've built CloudQuery as an open source high performance ELT framework so you should get pretty good results syncing all your cloud assets from high number of accounts (we have users syncing more than 10K Azure subscription and thousands of AWS accounts concurrently).

In a prior post I showed how to install Steampipe in AWS CloudShell to instantly query over 460+ resource types from your AWS APIs using SQL, and another post on how to use the Steampipe AWS Compliance mod to assess over 25+ security benchmarks across your AWS accounts.

The Steampipe AWS Compliance mod, is packed with hundreds of controls that check your AWS accounts for compliance with 25 benchmarks including NIST, PCI, HIPAA, SOC2, FedRAMP and more, now includes new controls for AWS CIS v2.0. If you're new to Steampipe, download Steampipe, install and configure the AWS plugin, and run these commands.

The Steampipe AWS Compliance mod, is packed with hundreds of open source controls that evaluate your AWS accounts for compliance with 25 benchmarks (NIST, PCI, HIPAA, etc). The mod now also includes new controls for AWS CIS v2.0.

The Steampipe AWS Compliance mod has more coverage with 21 security benchmarks (e.g. CIS v1.5, PCI, Foundations, NIST 800-53 rev5, etc).

AWS Compliance mod: includes instant compliance reports for CIS, PCI, NIST, HIPAA, SOC2 & more: https://hub.steampipe.io/mods/turbot/aws_compliance

For example, the AWS Compliance Mod layers benchmarks and controls covering 13 compliance standards including CIS, HIPAA, NIST, PCI, FedRAMP, SOC 2 and more. Each benchmark includes a set of pass/fail controls. Each control tests for a compliance recommendation such as "EC2 instances" should be managed by AWS Systems Manager" and reports OK or Alarm.

Steampipe (https://steampipe.io) is an open source CLI to query AWS and more with SQL. The AWS Compliance mod (https://hub.steampipe.io/mods/turbot/aws_compliance) has over 500 controls out of the box covering CIS, PCI, NIST, HIPAA, SOC2 & more you can run in your terminal across all your AWS accounts.