sqlglot VS go-mysql-server

https://github.com/tobymao/sqlglot :

> SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine [written in Python] . It can be used to format SQL or translate between 24 different dialects like DuckDB, Presto / Trino, Spark / Databricks, Snowflake, and BigQuery. It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects.

Thanks for the suggestion! I am using https://github.com/tobymao/sqlglot, which magically supports most SQL dialects. And yes, support for DuckDB is also in future plans

I recommend anyone who works with databases to write a simple engine. It's a lot simpler than you may think and it's a great exercise. If using python, sqlglot (https://github.com/tobymao/sqlglot) let's you skip all the parsing and it even does some simple optimizations. From the parsed query tree it's pretty straightforward to build a logical plan and execute that. You can even use python's builtin ast module to convert sql expressions into python ones (so no need for a custom interpreter!)

This could be a nice option to add sqlglot here. As an advanced sql parsing library.

You can compose SQL with https://ibis-project.org/tutorials/ibis-for-sql-users, which is using https://github.com/tobymao/sqlglot to parse the SQL under the hood.

As an alternative to parsing the SQL yourself, DuckDB's [relational API](https://duckdb.org/docs/api/python/relational_api) allows you to compose SQL expressions efficiently and lazily, which I've used when playing around with thinks like https://gist.github.com/ajfriend/eea0795546c7c44f1c24ab0560a...

SQLFluff is a linter formatter written in Python https://sqlfluff.com/

Heard there is a faster one called sqruff, but I have not tried it

https://www.quary.dev/blog/sqruff-launch

Might also be able to cajole sqlglot into being a formatter but it's designed for forgiving dialect conversion, not formatting

https://github.com/tobymao/sqlglot

This is probably referring to "zero changes to your driver code" and not "zero changes to the SQL you send over this driver".

Translating between SQL dialects is notoriously hard and attempts to translate [1] are working in 95% of cases. But the last 5% would require 5x amount of work. That's because "SQL dialect" also includes weird edge cases of type inference of things like COALESCE(5, FALSE) and emulation of system catalogs (pg_catalog, information_schema).

[1] https://github.com/tobymao/sqlglot

Recommend checking out https://github.com/tobymao/sqlglot if you are interested in this capability for other SQL dialects

Tools like this are helpful for:

- Rendering SQL in a consistent way, eg for snapshot testing

We implemented a query optimizer with a flexible intermediate representation in pure Go:

https://github.com/dolthub/go-mysql-server

Getting the IR correct so that it's both easy to use and flexible enough to be useful is a really interesting design challenge. Our primary abstraction in the query plan is called a Node, and is way more general than the IR type described in the article from OP. This has probably hurt us: we only recently separated the responsibility to fetch rows into its own part of the runtime, out of the IR -- originally row fetching was coupled to the Node type directly.

This is also the query engine that Dolt uses:

https://github.com/dolthub/dolt

But it has a plug-in architecture, so you can use the engine on any data source that implements a handful of Go interface.

Might be interested in https://github.com/dolthub/go-mysql-server, which also does this

https://github.com/dolthub/go-mysql-server

One item under "Scope of this project":

Provide a runnable server speaking the MySQL wire protocol, connected to data sources of your choice.

I just wanted to say thanks for https://github.com/dolthub/go-mysql-server

This is incredibly useful for anyone who wants to build their own DB or wrap another datasource so it's queryable via MySQL protocol.

a very cool project they also maintain is a MySQL server framework for arbitrary backends (in Go): https://github.com/dolthub/go-mysql-server

You can define a "virtual" table (schema, how to retrieve rows/columns) and then a MySQL client can connect and execute arbitrary queries on your table (which could just be an API or other source)