quix VS jupysql

Aerospike Data Browser is basically a stack that consists of Quix, Presto, and the Aerospike Connector for Presto, and is dockerized. Figure 1 depicts an under the hood view. The Quix UI provides a DB Explorer and a SQL editor, in addition to a notebook manager for managing your notebooks. Presto exposes a JDBC interface to Quix and uses the Aerospike Connector to translate SQL queries into API calls to the DB. Building a stack with the aforementioned components for a desktop installation is not trivial by any means. Presto can scale to 100’s of nodes for a large scale deployment, but we wanted to limit the data browser to a single Presto instance that would run both the coordinator and worker in the developer's desktop environment. Our initial size of the Presto docker image was over 2GB, which was not acceptable. Hence, we stripped out all but the Aerospike connector from the plugin directory. Similarly, we had to downsize the Quix connector. Finally, we got the compressed docker image size under 1GB. We also made a design decision to default to schema inference so that a user that does not know the schema apriori is not left out.

Hey, HN community!

We're stoked to launch JupySQL today! JupySQL is an open-source library that brings a modern SQL experience to Jupyter. JupySQL is compatible with all major databases, such as Snowflake, Redshift, PostgreSQL, MySQL, MariaDB, DuckDB, SQL Server, Clickhouse, Trino, and more!

To get started, check out our tutorial: https://jupysql.ploomber.io/en/latest/quick-start.html

SQL is the defacto language for data analysis; however, analysis often requires a mix of SQL and Python. JupySQL bridges this gap, allowing users to execute SQL queries seamlessly in Jupyter and continue their analysis in Python. Add %%sql to the top of your cell and start writing SQL.

Here are some of JupySQL's main features:

- Syntax highlighting

Please show your support with a 🌟: https://github.com/ploomber/jupysql

If anyone is looking for something like this in Python/Jupyter, check out JupySQL: https://github.com/ploomber/jupysql

How are people managing the existence of data frame APIs like pandas/polars with SQL engines like BigQuery, Snowflake, and DuckDB?

Most of my notebooks are a mix of SQL and Python: SQL for most processing, dump the results as a pandas dataframe (via https://github.com/ploomber/jupysql) and then use Python for operations that are difficult to express with SQL (or that I don't know how to do it), so I end up with 80% SQL, 20% Python.

Unsure if this is the best workflow but it's the most efficient one I've come up with.

Disclaimer: my team develops JupySQL.