opteryx VS sqlglot

Thanks for sharing.

I have a SQL Engine in Python too (https://github.com/mabel-dev/opteryx). I focused my initial effort on supporting SQL statements and making the usage feel like a database - that probably reflects the problem I had in front of me when I set out - only handling handfuls of gigabytes in a batch environment for ETLs with a group of new-to-data-engineering engineers. Have recently started looking more at real-time performance, such as distributing work. Am interesting in how you've approached.

This is probably referring to "zero changes to your driver code" and not "zero changes to the SQL you send over this driver".

Translating between SQL dialects is notoriously hard and attempts to translate [1] are working in 95% of cases. But the last 5% would require 5x amount of work. That's because "SQL dialect" also includes weird edge cases of type inference of things like COALESCE(5, FALSE) and emulation of system catalogs (pg_catalog, information_schema).

[1] https://github.com/tobymao/sqlglot

Recommend checking out https://github.com/tobymao/sqlglot if you are interested in this capability for other SQL dialects

Tools like this are helpful for:

- Rendering SQL in a consistent way, eg for snapshot testing

sqlglot – Python SQL Parser and Transpiler

In the project we used Python lib networkx and a DiGraph object (Direct Graph). To detect a table reference in a Query, we use sqlglot, a SQL parser (among other things) that works well with Bigquery.

Cool! Is this built with sqlglot[1] on the back end?

[1] https://github.com/tobymao/sqlglot

Wanted to give sqlglot a shoutout as it saved me a ton of time.