| jupysql | tpch | |
|---|---|---|
| 8 | 5 | |
| 605 | 56 | |
| 4.6% | - | |
| 9.1 | 8.4 | |
| 21 days ago | 7 days ago | |
| Python | C | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
jupysql
-
Show HN: JupySQL – a SQL client for Jupyter (ipython-SQL successor)
Hey, HN community!
We're stoked to launch JupySQL today! JupySQL is an open-source library that brings a modern SQL experience to Jupyter. JupySQL is compatible with all major databases, such as Snowflake, Redshift, PostgreSQL, MySQL, MariaDB, DuckDB, SQL Server, Clickhouse, Trino, and more!
To get started, check out our tutorial: https://jupysql.ploomber.io/en/latest/quick-start.html
SQL is the defacto language for data analysis; however, analysis often requires a mix of SQL and Python. JupySQL bridges this gap, allowing users to execute SQL queries seamlessly in Jupyter and continue their analysis in Python. Add %%sql to the top of your cell and start writing SQL.
Here are some of JupySQL's main features:
- Syntax highlighting
-
JupySQL: Connecting to a SQL database from Jupyter
Please show your support with a 🌟: https://github.com/ploomber/jupysql
- GitHub - ploomber/jupysql: Better SQL in Jupyter. 📊
- SQL CTE's in Jupyter notebooks, DuckDB integration and more
- TL;DR incorporate SQL functionality within Jupyter, access to modern data processing DBs (like DuckDB), polars and data exploration through plotting easier with JupySQL.
-
Evidence – Business Intelligence as Code
If anyone is looking for something like this in Python/Jupyter, check out JupySQL: https://github.com/ploomber/jupysql
- A full-featured SQL client for Jupyter
-
Pandas v2.0 Released
How are people managing the existence of data frame APIs like pandas/polars with SQL engines like BigQuery, Snowflake, and DuckDB?
Most of my notebooks are a mix of SQL and Python: SQL for most processing, dump the results as a pandas dataframe (via https://github.com/ploomber/jupysql) and then use Python for operations that are difficult to express with SQL (or that I don't know how to do it), so I end up with 80% SQL, 20% Python.
Unsure if this is the best workflow but it's the most efficient one I've come up with.
Disclaimer: my team develops JupySQL.
tpch
- pandas 2.0 is coming out soon
-
Pandas 2.0 Released
I did run the benchmarks on TPC-H: https://github.com/pola-rs/tpch/pull/36
-
Pandas v2.0 Released
Polars author here. I have run the TPC-H benchmark against polars and pandas 2.0 backed by arrow types.
https://github.com/pola-rs/tpch/pull/36
Pandas having arrow as backend is great and will make interop with the arrow community (and polars) much better.
However, if you need performance, polars remains orders of magnitudes faster on whole queries, changing to the arrow memory format does not change that.
- Polars vs Pandas
-
From NumPy to Arrow: How Pandas 2.0 is Changing Data Processing for the Better
/u/ritchie46 maintains a repo using the more realistic TPC-H benchmarks. He just merged a PR with the pandas-back-by-arrow numbers (https://github.com/pola-rs/tpch/pull/36) and still, `polars` is miles ahead in terms of performance.
What are some alternatives?
grai-core
datapane - Build and share data reports in 100% Python
nba-monte-carlo - Monte Carlo simulation of the NBA season, leveraging dbt, duckdb and evidence.dev
chdb-server-bak - API Server for chDB, an in-process SQL OLAP Engine powered by ClickHouse
pytest-mock-resources - Pytest Fixtures that let you actually test against external resource (Postgres, Mongo, Redshift...) dependent code.
prism - Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.
dbt-ml-preprocessing - A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.
portable-data-stack-dagster - A portable Datamart and Business Intelligence suite built with Docker, Dagster, dbt, DuckDB, PostgreSQL and Superset
mais - ⚙️ Código de manutenção do datalake (metadados e pacotes de acesso) | 📖 Docs: https://basedosdados.github.io/mais/
Pandas - Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
snowpark-python - Snowflake Snowpark Python API