steampipe
trustfall
steampipe | trustfall | |
---|---|---|
146 | 24 | |
6,412 | 2,273 | |
1.1% | - | |
9.7 | 9.2 | |
2 days ago | 12 days ago | |
Go | Rust | |
GNU Affero General Public License v3.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
steampipe
- Steampipe: Dynamically query APIs, code and more with SQL
-
Cloud Tools You Probably Haven't Heard Of
Steampipe is a tool for querying cloud APIs and other data sources using SQL in a zero-ETL manner.
-
Show HN: Query Your Sheets with SheetSQL
Readers may also enjoy Steampipe [1], an open source CLI to live query Google Sheets [2] and 140+ other services with SQL (e.g. AWS, GitHub, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc across the services. (Disclaimer - I'm a lead on the project.)
1 - https://github.com/turbot/steampipe
-
Osquery: An sqlite3 virtual table exposing operating system data to SQL
be mindful of its AGPLv3 https://github.com/turbot/steampipe/blob/v0.21.8/LICENSE (AFAIK v0.4.3 is the last MIT release https://github.com/turbot/steampipe/blob/v0.4.3/LICENSE ) and the actual providers are Apache 2 <https://github.com/turbot/steampipe-plugin-aws/blob/v0.131.0...> (but I don't know if provider drift makes them compatible with 0.4 or not)
iasql seems to be AWS only, but good for them for taking this on:
-
How to run an AWS CIS v3.0 assessment in CloudShell
In a prior post I showed how to install Steampipe in AWS CloudShell to instantly query over 460+ resource types from your AWS APIs using SQL, and another post on how to use the Steampipe AWS Compliance mod to assess over 25+ security benchmarks across your AWS accounts.
- Git Query Language
- Query Cloud and SaaS APIs with SQL
-
Cutting down AWS cost by $150k per year simply by shutting things off
Readers may find Steampipe's [1] AWS Thrifty Mod [2] useful. It will automatically scan multiple accounts and regions for 50 cost saving opportunities - many of which are looking for over-provisioned or unused resources. For example, it's crazy how much you can save by doing things like just converting your EBS volumes to the newer gp3 type. Combine with Flowpipe [3] to automate checks and actions. It's all open source and extensible.
1 - https://github.com/turbot/steampipe
- FLaNK Weekly 08 Jan 2024
-
Zero-ETL for Postgres: Live-query cloud APIs with 100 open source FDWs
Steampipe [1] is an open source project [2] that includes an embedded Postgres to instantly query cloud, code & more with SQL. This release expands our plugin ecosystem [3] to be a full Zero-ETL platform. Steampipe plugins can now run natively in your own Postgres as Foreign Data Wrappers [4], as SQLite extensions [5] or as simple data export tools [6]. Please give it a try, we'd love your feedback and contributions!
1 - https://steampipe.io
trustfall
-
Planner programming blows my mind
Pleasantly surprised to see Predrag show up as a reviewer, but at the same time not at all surprised:
- The [Firebase technical screen](https://startupandrew.com/posts/how-firebase-interviewed-sof...) would have been much easier with something like this, as it was Just Another Optimization Problem™. Part of me wants to try it again with Picat!
- He's doing other very interesting things with programming languages, e.g.: https://github.com/obi1kenobi/trustfall
-
Ask HN: Who wants to be hired? (December 2023)
Email: [email protected]
Hi, I'm u9g. I'm looking for an internship for either this winter (in a few weeks) or a summer internship!
Some things I've done:
I integrated a [query engine for lints](https://github.com/obi1kenobi/trustfall) (and contributed optimizations) into [OXC](https://github.com/oxc-project/oxc) (a new Rust-based Javascript Linter).
I wrote [several](https://github.com/u9g/money-lens) [toy](https://github.com/u9g/quickquestion/tree/main/extension) languages and syntax highlighting for them.
I also wrote a [js-to-scheme transpiler](https://github.com/u9g/js2scheme/blob/main/example.js).
Happy to work on things compiler or database oriented, but also happy to learn something new!
-
Semver violations are common, better tooling is the answer
If you'd like to dig deeper, here are some links:
- cargo-semver-checks: https://github.com/obi1kenobi/cargo-semver-checks
- Trustfall query engine, which powers cargo-semver-checks: https://github.com/obi1kenobi/trustfall
- Trustfall playground, where you can query Rust library APIs in your browser -- for example, "which structs in `itertools` are importable by more than one path": https://play.predr.ag/rustdoc#?f=2&q=*3-Structs-importable-f...
- 10min conference talk on Trustfall: https://www.hytradboi.com/2022/how-to-query-almost-everythin...
I'm also giving a talk at P99 CONF in a few months about how Trustfall's new optimizations API made cargo-semver-checks over 2300x faster: https://twitter.com/PredragGruevski/status/16893002495908003...
-
Cargo-semver-checks: Scan your Rust crate for semver violations
No worries!
Trustfall is even more general than semver actually, one can use it to query any data source -- even HN itself:
https://play.predr.ag/HackerNews
Here's a link to that project if you'd like to check it out:
https://github.com/obi1kenobi/trustfall
I gave a 10min conference talk on it last year titled "How to query (almost) everything"
https://www.hytradboi.com/2022/how-to-query-almost-everythin...
-
We created Taxi, and why we felt the need for Another Schema Language
I've had my eye on Taxi for a while, and it's neat! I agree that the problems it aims to solve are real and painful in real life.
In my experience, I'm not sure people care about schemas or schema languages — they are just implementation details best left under the hood. This is why in my own work, I started on the query end of the spectrum instead.
This is why I built Trustfall, a query engine able to query any data source: DBs, APIs, files like PDF, CSV, or JSON, or any combination of those.
GitHub: https://github.com/obi1kenobi/trustfall
Playground showing the query "which GitHub/Twitter users comment on HackerNews stories about OpenAI?": https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVl...
- cargo-semver-checks v0.20 and Trustfall v0.4 released — semver-check up to 2354x faster
- DuckDB: Querying JSON files as if they were tables
-
Speeding up Rust semver-checking by over 2000x
Here's an example adapter for querying RSS/Atom feeds. Here's an example adapter for querying the HackerNews APIs. Both projects also include a schema and example queries which you can run.
- Trustfall: How to Query (Almost) Everything
What are some alternatives?
cloudquery - The open source high performance ELT framework powered by Apache Arrow
postgrest - REST API for any Postgres database
cloud-custodian - Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources
cargo-semver-checks-action - A GitHub Action for running cargo-semver-checks
metriql - The metrics layer for your data. Join us at https://metriql.com/slack
join-monster - A GraphQL to SQL query execution layer for query planning and batch data fetching.
inspec-aws - InSpec AWS Resource Pack https://www.inspec.io/
graphql-java - GraphQL Java implementation
steampipe-mod-github-sherlock - Interrogate your GitHub resources with the help of the world's greatest detectives: Powerpipe + Steampipe + Sherlock.
dgs-framework - [Moved to: https://github.com/Netflix/dgs-framework]
embedded-postgres-binaries - Lightweight bundles of PostgreSQL binaries with reduced size intended for testing purposes.
genql - Type safe TypeScript client for any GraphQL API