cloudquery
steampipe
Our great sponsors
cloudquery | steampipe | |
---|---|---|
102 | 144 | |
5,524 | 6,325 | |
2.6% | 4.2% | |
10.0 | 9.7 | |
6 days ago | 7 days ago | |
Go | Go | |
Mozilla Public License 2.0 | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cloudquery
-
Cloud asset tracking
There both do something like what you're looking for.... https://github.com/cloudquery/cloudquery https://github.com/openraven/magpie
-
Show HN: Nango – Open unified API for product integrations
Unified API is a holly grail but as many said quite difficult to abstract every use case in a scalable way that won't break. At CloudQuery (https://github.com/cloudquery/cloudquery) we focus solely on the ELT use-case(Founder/Maintainer here).
-
Welcome to Datasette Cloud
Congrats!! How does it compare to the ELT space and the modern data stack where you have ingestion/storage/visualization layers decoupled?
Asking as the founder of CloudQuery (https://github.com/cloudquery/cloudquery), Saw Datasette quite a few times around data exploration but curious to hear about the most popular use-cases of Datasette!
-
Launch HN: PeerDB (YC S23) – Fast, Native ETL/ELT for Postgres
Congrats!! We also focus on performance at CloudQuery (https://github.com/cloudquery/cloudquery) by using Golang, gRPC and still trying to be abstract enough to support different databases :)
In any case good luck!
-
airbyte VS cloudquery - a user suggested alternative
2 projects | 2 Jun 2023
CloudQuery for ETL
2 projects | 2 Jun 2023Another ELT framework that's an alternative to Airbyte
-
meltano VS cloudquery - a user suggested alternative
2 projects | 2 Jun 2023
Another alternate ELT
-
Cloudquery, Resoto, Steampipe, or Airbyte?
Cloudquery: https://cloudquery.io/
Hello! Im Yevgeny, Founder & maintainer at CloudQuery . We've built CloudQuery as an open source high performance ELT framework so you should get pretty good results syncing all your cloud assets from high number of accounts (we have users syncing more than 10K Azure subscription and thousands of AWS accounts concurrently).
-
Test On 4 Concurrent Jobs Using Python-Polars 0.17.11 to GroupBy Billion Rows
CloudQuery supports a lot of APIs https://github.com/cloudquery/cloudquery
steampipe
-
Show HN: Query Your Sheets with SheetSQL
Readers may also enjoy Steampipe [1], an open source CLI to live query Google Sheets [2] and 140+ other services with SQL (e.g. AWS, GitHub, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc across the services. (Disclaimer - I'm a lead on the project.)
-
Osquery: An sqlite3 virtual table exposing operating system data to SQL
be mindful of its AGPLv3 https://github.com/turbot/steampipe/blob/v0.21.8/LICENSE (AFAIK v0.4.3 is the last MIT release https://github.com/turbot/steampipe/blob/v0.4.3/LICENSE ) and the actual providers are Apache 2 <https://github.com/turbot/steampipe-plugin-aws/blob/v0.131.0...> (but I don't know if provider drift makes them compatible with 0.4 or not)
iasql seems to be AWS only, but good for them for taking this on:
Few projects in the same realm that you should also checkout -
[1] Steampipe (https://steampipe.io/)
[2] InfraSQL (https://iasql.com/)
-
How to run an AWS CIS v3.0 assessment in CloudShell
In a prior post I showed how to install Steampipe in AWS CloudShell to instantly query over 460+ resource types from your AWS APIs using SQL, and another post on how to use the Steampipe AWS Compliance mod to assess over 25+ security benchmarks across your AWS accounts.
- Git Query Language
-
Cutting down AWS cost by $150k per year simply by shutting things off
Readers may find Steampipe's [1] AWS Thrifty Mod [2] useful. It will automatically scan multiple accounts and regions for 50 cost saving opportunities - many of which are looking for over-provisioned or unused resources. For example, it's crazy how much you can save by doing things like just converting your EBS volumes to the newer gp3 type. Combine with Flowpipe [3] to automate checks and actions. It's all open source and extensible.
- FLaNK Weekly 08 Jan 2024
-
Zero-ETL for Postgres: Live-query cloud APIs with 100 open source FDWs
Steampipe [1] is an open source project [2] that includes an embedded Postgres to instantly query cloud, code & more with SQL. This release expands our plugin ecosystem [3] to be a full Zero-ETL platform. Steampipe plugins can now run natively in your own Postgres as Foreign Data Wrappers [4], as SQLite extensions [5] or as simple data export tools [6]. Please give it a try, we'd love your feedback and contributions!
-
5 Open Source tools written in Golang that you should know about
Steampipe is a tool that simplifies data extraction from APIs and services, eliminating the need for ETL (Extract, Transform, Load) processes. It includes several components: the Steampipe CLI for querying APIs, ensuring compliance, and creating dashboards; Steampipe Postgres FDWs (Foreign Data Wrappers) which turn APIs into foreign tables in Postgres; Steampipe SQLite extensions that convert APIs into SQLite virtual tables; and standalone export tools for directly exporting data from APIs without requiring a database. Additionally, Turbot Pipes, a part of Steampipe, offers capabilities for querying, checking, and visualizing data, tailored for DevOps teams with a focus on intelligence, automation, and security.
- Pipe Dreams: The life and times of Yahoo Pipes
What are some alternatives?
steampipe-mod-aws-compliance - Run individual controls or full compliance benchmarks for CIS, PCI, NIST, HIPAA and more across all of your AWS accounts using Powerpipe and Steampipe.
cloud-custodian - Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources
cloudsploit - Cloud Security Posture Management (CSPM)
cartography - Cartography is a Python tool that consolidates infrastructure assets and the relationships between them in an intuitive graph view powered by a Neo4j database.
opencspm - Open Cloud Security Posture Management Engine
inspec-aws - InSpec AWS Resource Pack https://www.inspec.io/
metriql - The metrics layer for your data. Join us at https://metriql.com/slack
steampipe-mod-github-sherlock - Interrogate your GitHub resources with the help of the world's greatest detectives: Powerpipe + Steampipe + Sherlock.
steampipe-plugin-oci - Use SQL to instantly query Oracle Cloud resources across regions and accounts. Open source CLI. No DB required.
Typesense - Open Source alternative to Algolia + Pinecone and an Easier-to-Use alternative to ElasticSearch ⚡ 🔍 ✨ Fast, typo tolerant, in-memory fuzzy Search Engine for building delightful search experiences
cue - CUE has moved to https://github.com/cue-lang/cue