ibis
steampipe
Our great sponsors
ibis | steampipe | |
---|---|---|
22 | 146 | |
4,041 | 6,366 | |
7.2% | 2.2% | |
10.0 | 9.7 | |
7 days ago | 8 days ago | |
Python | Go | |
Apache License 2.0 | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ibis
-
This Week In Python
ibis – portable Python dataframe library
- Ibis: The portable Python dataframe library
- FLaNK Stack 26 February 2024
-
Quarto
The main benefit is that you get a Python (or R, Julia or Rust) interpreter. So you can evaluate code. A good example of the value of this is the Ibis docs which use Quarto: https://ibis-project.org/
-
Polars – A bird's eye view of Polars
Ive found polars quite intuitive, though for python, I lean more towards [ibis](https://ibis-project.org/). The interface is nearly identical, but ibis has the benefit if building sql queries before pulling any actual data (like dbplyr) — whereas polars requires the data to be in-memory (at least for rdb’s, though correct me if Im wrong)
this to me seems like a good argument for only using ibis, but Im happy to be convinced otherwise
- Ibis – Universal Interface for Data Wrangling
-
Vanna.ai: Chat with your SQL database
Please add Ibis Birdbrain https://ibis-project.github.io/ibis-birdbrain/ to the list. Birdbrain is an AI-powered data bot, built on Ibis and Marvin, supporting more than 18 database backends.
See https://github.com/ibis-project/ibis and https://ibis-project.org for more details.
- Ibis
-
How to Use Pandas for Data Analysis
Ibis: The portable Python dataframe library
steampipe
- Steampipe: Dynamically query APIs, code and more with SQL
-
Cloud Tools You Probably Haven't Heard Of
Steampipe is a tool for querying cloud APIs and other data sources using SQL in a zero-ETL manner.
-
Show HN: Query Your Sheets with SheetSQL
Readers may also enjoy Steampipe [1], an open source CLI to live query Google Sheets [2] and 140+ other services with SQL (e.g. AWS, GitHub, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc across the services. (Disclaimer - I'm a lead on the project.)
-
Osquery: An sqlite3 virtual table exposing operating system data to SQL
be mindful of its AGPLv3 https://github.com/turbot/steampipe/blob/v0.21.8/LICENSE (AFAIK v0.4.3 is the last MIT release https://github.com/turbot/steampipe/blob/v0.4.3/LICENSE ) and the actual providers are Apache 2 <https://github.com/turbot/steampipe-plugin-aws/blob/v0.131.0...> (but I don't know if provider drift makes them compatible with 0.4 or not)
iasql seems to be AWS only, but good for them for taking this on:
-
How to run an AWS CIS v3.0 assessment in CloudShell
In a prior post I showed how to install Steampipe in AWS CloudShell to instantly query over 460+ resource types from your AWS APIs using SQL, and another post on how to use the Steampipe AWS Compliance mod to assess over 25+ security benchmarks across your AWS accounts.
- Git Query Language
- Query Cloud and SaaS APIs with SQL
-
Cutting down AWS cost by $150k per year simply by shutting things off
Readers may find Steampipe's [1] AWS Thrifty Mod [2] useful. It will automatically scan multiple accounts and regions for 50 cost saving opportunities - many of which are looking for over-provisioned or unused resources. For example, it's crazy how much you can save by doing things like just converting your EBS volumes to the newer gp3 type. Combine with Flowpipe [3] to automate checks and actions. It's all open source and extensible.
- FLaNK Weekly 08 Jan 2024
-
Zero-ETL for Postgres: Live-query cloud APIs with 100 open source FDWs
Steampipe [1] is an open source project [2] that includes an embedded Postgres to instantly query cloud, code & more with SQL. This release expands our plugin ecosystem [3] to be a full Zero-ETL platform. Steampipe plugins can now run natively in your own Postgres as Foreign Data Wrappers [4], as SQLite extensions [5] or as simple data export tools [6]. Please give it a try, we'd love your feedback and contributions!
What are some alternatives?
snowflake-connector-python - Snowflake Connector for Python
cloudquery - The open source high performance ELT framework powered by Apache Arrow
PySpark-Boilerplate - A boilerplate for writing PySpark Jobs
cloud-custodian - Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources
Apache Impala - Apache Impala
metriql - The metrics layer for your data. Join us at https://metriql.com/slack
pangres - SQL upsert using pandas DataFrames for PostgreSQL, SQlite and MySQL with extra features
inspec-aws - InSpec AWS Resource Pack https://www.inspec.io/
sqlite_scanner - DuckDB extension to read and write to SQLite databases
steampipe-mod-github-sherlock - Interrogate your GitHub resources with the help of the world's greatest detectives: Powerpipe + Steampipe + Sherlock.
katacoda
embedded-postgres-binaries - Lightweight bundles of PostgreSQL binaries with reduced size intended for testing purposes.