cloudquery VS cloud-pricing-api

Check out CloudQuery - https://github.com/cloudquery/cloudquery for an easy cloud asset inventory.

There both do something like what you're looking for.... https://github.com/cloudquery/cloudquery https://github.com/openraven/magpie

Unified API is a holly grail but as many said quite difficult to abstract every use case in a scalable way that won't break. At CloudQuery (https://github.com/cloudquery/cloudquery) we focus solely on the ELT use-case(Founder/Maintainer here).

Congrats!! How does it compare to the ELT space and the modern data stack where you have ingestion/storage/visualization layers decoupled?

Asking as the founder of CloudQuery (https://github.com/cloudquery/cloudquery), Saw Datasette quite a few times around data exploration but curious to hear about the most popular use-cases of Datasette!

Congrats!! We also focus on performance at CloudQuery (https://github.com/cloudquery/cloudquery) by using Golang, gRPC and still trying to be abstract enough to support different databases :)

In any case good luck!

CloudQuery for ETL

Another ELT framework that's an alternative to Airbyte

Another alternate ELT

Check out CloudQuery, we have PostgreSQL source connectors and S3 destination that supports parquet (Disclaimer: Maintainer and founder here)

Hello! Im Yevgeny, Founder & maintainer at CloudQuery . We've built CloudQuery as an open source high performance ELT framework so you should get pretty good results syncing all your cloud assets from high number of accounts (we have users syncing more than 10K Azure subscription and thousands of AWS accounts concurrently).

u/ErikHumphrey yep, by default Infracost connects to our hosted Cloud Pricing API but you can self-host that part (which fetches the 3M+ prices from AWS+Azure+Google and self-updates), see https://www.infracost.io/docs/cloud_pricing_api/self_hosted/ and the "DISABLE_TELEMETRY" env variable that is mentioned in https://github.com/infracost/cloud-pricing-api

Infracost has two versions: a VSCode addon and a command line program. Both do the same thing: parse Terraform code, pull the current cost price points from a cloud pricing API, and output an estimate.

The complexity is crazy for sure. Around 10 years ago I started looking at cloud costs as part of my PhD - back then there were around 10,000 price points that I was scraping manually, today there are more than 3 million prices between AWS, Azure and Google that we're fetching in https://github.com/infracost/cloud-pricing-api

yup, checkout this example. Happy to help with the other filters you might need, I spend a lot of time exploring pricing params