parser VS zetasql

For MySQL the situation is a bit different. sqlc uses the parser of TiDB (https://github.com/pingcap/parser), which is a parser that aims to be basically compatible with MySQL, but is quite young and is not a MySQL parser. The most basic queries work, but even simple joins or aggregations usually result in variables with unknown data types or wrong nullability. So you loose a lot of the benefits of sqlc. Manual type annotations for MySQL also do not work most of the time. They are simply ignored and forwarded to MySQL as invalid query if they do not occur on a place where sqlc is expecting them.

This is a disaster waiting to happen. Regular expressions should never be used to parse non-regular languages, of which SQL is one.

There are a variety of mature MySQL dialect parsers available[1][2], and MySQL should have its own public APIs for transforming a query into an AST. Any of those would be a safer and more correct alternative.

[1]: https://github.com/pingcap/parser

[2]: https://github.com/square/mysql-parser

Pingcap parser is a MySQL parser in Go.

There are even table-valued functions.

These things are not widespread, and differ by implementation, and the way these are used by clients are copy-and-paste. Something as thoughtful as ZetaSQL https://github.com/google/zetasql does not have mechanisms for structuring (modules, packages, interfaces). SQL will not, cannot evolve into such a direction (or, anything that evolves, will not be recognizable as SQL).

Hi, I develop a BigQuery emulator ( https://github.com/goccy/bigquery-emulator ) from early 2022. It is written in Go, but can be used from bq command line tool and other language's (e.g. Python ) client SDK after installing docker image or released binary. It currently supports over 200 of the nearly 330 standard functions in BigQuery and all data types except GEOGRAPHY ( see https://github.com/goccy/go-zetasqlite#status for details ). ZetaSQL ( https://github.com/google/zetasql ) is used to parse and analyze queries.

We are using a Python client binding for ZetaSQL GRPC local service in our application to analyze statements and extract referenced tables and output columns.

If you don't want to do it yourself, there's this:

https://github.com/google/zetasql

Parsing is huge but it's amazing how small a part of the job it is. This library isn't even the half of it.

I mean the postgres parser (and semantic changes) for ZetaSQL. The zetasql parser is in a file called zetasql/parser/bison_parser.y, I strongly suspect they now have a file called something like zetasql/pgparser/bison_parser.y as well (and much more pervasive changes to support the deeper differences in the dialects).

This is the lexical structure and syntax docs for the new postgres inteface to cloud spanner:

https://cloud.google.com/spanner/docs/postgresql/lexical

And this is the zetasql lexical structure and syntax docs:

https://github.com/google/zetasql/blob/master/docs/lexical.m...

Notice that the new PG docs are an edit of the Zeta ones - evidence that my hypothesis is correct.

zetasql implements BigQuery, Spanner, and Dataflow dialects.