cloudquery VS policy_sentry

There both do something like what you're looking for.... https://github.com/cloudquery/cloudquery https://github.com/openraven/magpie

Unified API is a holly grail but as many said quite difficult to abstract every use case in a scalable way that won't break. At CloudQuery (https://github.com/cloudquery/cloudquery) we focus solely on the ELT use-case(Founder/Maintainer here).

Congrats!! How does it compare to the ELT space and the modern data stack where you have ingestion/storage/visualization layers decoupled?

Asking as the founder of CloudQuery (https://github.com/cloudquery/cloudquery), Saw Datasette quite a few times around data exploration but curious to hear about the most popular use-cases of Datasette!

Congrats!! We also focus on performance at CloudQuery (https://github.com/cloudquery/cloudquery) by using Golang, gRPC and still trying to be abstract enough to support different databases :)

In any case good luck!

CloudQuery for ETL

Another ELT framework that's an alternative to Airbyte

Another alternate ELT

Cloudquery: https://cloudquery.io/

Hello! Im Yevgeny, Founder & maintainer at CloudQuery . We've built CloudQuery as an open source high performance ELT framework so you should get pretty good results syncing all your cloud assets from high number of accounts (we have users syncing more than 10K Azure subscription and thousands of AWS accounts concurrently).

CloudQuery supports a lot of APIs https://github.com/cloudquery/cloudquery

All of the sdks support client side monitoring (CSM), so these sort of tools can be built client side. https://boto3.amazonaws.com/v1/documentation/api/1.10.46/gui...

afaics the only challenge is mapping some of the apis to iam as its only 85% 1:1

There's also tools for helping with iam like (generator, and linter)

https://github.com/salesforce/policy_sentry

https://github.com/duo-labs/parliament

It is an open source initiative from Salesforce. Using Policy Sentry, it is easy to automate the creation of IAM policies with little knowledge on security.

You can use tools like policy_sentry to create least privilege IAM policies.

The tool is in a similar space to iamlive, policy_sentry, and consoleme (all of which are worth checking out too if you're interested in making AWS security easier) but the main points of difference I see are:

@kmcquade ur awesome ! we are users of https://github.com/salesforce/policy_sentry and definitely definitely https://github.com/salesforce/cloudsplaining .

If I could give you guys money, I would. You should totally build a startup around it.