Our great sponsors
-
Excited to see Steampipe shared here - thanks kiyanwang! I'm a lead on the project, so sharing some quick info below and happy to answer any questions.
Steampipe is open source [1] and uses Postgres foreign data wrappers under the hood [2]. We have 84+ plugins to SQL query AWS, GitHub, Slack, HN, etc [3]. Mods (written in HCL) provide dashboards as code and automated security & compliance benchmarks [3]. We'd love your help & feedback!
-
steampipe-plugin-tailscale
Use SQL to instantly query Tailscale resources. Open source CLI. No DB required.
Glad AWS worked. Sorry you hit a bump with tailscale - we'd love feedback / contributions for that plugin [1], we just released it this month so may be edges for more complex configurations :-(
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
Seems like the next step past https://osquery.io/
-
prql
PRQL is a modern language for transforming data — a simple, powerful, pipelined SQL replacement
No mention of PRQL yet, it's an effort to tackle some of these shortcomings
-
steampipe-mod-aws-thrifty
Are you a Thrifty AWS dev? This mod checks your AWS accounts for unused and under-utilized resources using Powerpipe and Steampipe.
I feel your pain - but Steampipe won't solve this (yet). We have AWS Thrifty [1] to scan your env for cost savings and other tables to query your spend [2], but haven't tackled the 1.8GB of pricing JSON data into a table yet.
-
We have work in progress to support go-getter [1] allowing use of files from many different sources. There is a WIP branch [2] building that into the Steampipe Plugin SDK, so it can be immediately enabled for all our file based plugins (CSV, Terraform, etc) [3]. Watch this space and hopefully you can help us test!
-
steampipe-plugin-sdk
Steampipe Plugin SDK is a simple abstraction layer to write a Steampipe plugin. Plugins automatically work across all engine types including the Steampipe CLI, Postgres FDW, SQLite extension and the export CLI.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
steampipe-plugin-terraform
Use SQL to instantly query resources, data sources and more from Terraform code. Open source CLI. No DB required.
-
octosql
OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.
To add somewhat of a counterpoint to the other response, I've tried the Steampipe CSV plugin and got 50x slower performance vs OctoSQL[0], which is itself 5x slower than something like DataFusion[1]. The CSV plugin doesn't contact any external API's so it should be a good benchmark of the plugin architecture, though it might just not be optimized yet.
That said, I don't imagine this ever being a bottleneck for the main use case of Steampipe - in that case I think the APIs themselves will always be the limiting part. But it does - potentially - speak to what you can expect if you'd like to extend your usage of Steampipe to more than just DevOps data.
[0]: https://github.com/cube2222/octosql
[1]: https://github.com/apache/arrow-datafusion
Disclaimer: author of OctoSQL
-
To add somewhat of a counterpoint to the other response, I've tried the Steampipe CSV plugin and got 50x slower performance vs OctoSQL[0], which is itself 5x slower than something like DataFusion[1]. The CSV plugin doesn't contact any external API's so it should be a good benchmark of the plugin architecture, though it might just not be optimized yet.
That said, I don't imagine this ever being a bottleneck for the main use case of Steampipe - in that case I think the APIs themselves will always be the limiting part. But it does - potentially - speak to what you can expect if you'd like to extend your usage of Steampipe to more than just DevOps data.
[0]: https://github.com/cube2222/octosql
[1]: https://github.com/apache/arrow-datafusion
Disclaimer: author of OctoSQL
-
The ergonomics issue of SQL can be workaround by transpiler.
I do like SQL the syntax but it's hard to discard the ecosystem around SQL.
For instance, I made better-sql [1] recently which generate SQL from a language similar to GraphQL and EdgeDB query language.
