sqlite-parser
squawk
sqlite-parser | squawk | |
---|---|---|
3 | 5 | |
124 | 506 | |
- | - | |
0.0 | 6.1 | |
over 1 year ago | 7 days ago | |
PLpgSQL | Rust | |
MIT License | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sqlite-parser
-
Postgres Language Server: Implementing the Parser
Oof. Yes, I meant scraper.
> ...concerned about the times reported
Agreed.
FWIW...
My wall time for randtest1 is ~55ms. https://github.com/bkiers/sqlite-parser/blob/master/src/test...
And statements like these are < 20ms each.
select ((select 1) union (select 1));
-
SQLite Internals: How the Most Used Database Works
> ...than it would be to learn the exact syntax and quirks and possibly bugs of someone else's implementation...
Yup. Also, having deep knowledge of the language is required.
SQLite's grammar is neat. Creating a compatible parser would make a fun project. Here's a pretty good example: https://github.com/bkiers/sqlite-parser (Actual ANTLR 4 grammar: https://github.com/bkiers/sqlite-parser/blob/master/src/main... )
Postgres, which tries to be compliant with the latest standards, however...
SQL-2016 is a beast. Not to mention all the dialects.
I'm updating my personal (soon to be FOSS) grammar from ANTLR 3 LL(k) to ANTLR 4 ALL().
I've long had a working knowledge of SQL-92, with some SQL-1999 (eg common table expressions).
But the new structures and extensions are a bit overwhelming.
Fortunately, ANTLR project has ~dozen FOSS grammars to learn from. https://github.com/antlr/grammars-v4/tree/master/sql
They mostly mechanically translate BNFs to LL(k) with some ALL(). Meaning few take advantage of left-recursion. https://github.com/antlr/antlr4/blob/master/doc/left-recursi...
Honestly, I struggled to understand these grammars. Plus, not being conversant with the SQL-2016 was a huge impediment. Just finding a succinct corbis of test cases was a huge hurdle for me.
Fortunately, the H2 Database project is a great resource. https://github.com/h2database/h2database/tree/master/h2/src/...
Now for the exciting conclusion...
My ANTLR grammar which passes all of H2's tests looks nothing like any of the official or product specific BNFs.
Further, I found discrepancy between the product specific BNFs and their implementations.
So a lot of trial & error is required for a "real world" parser. Which would explain why the professional SQL parsing tools charge money.
I still think creating a parser for SQLite is a great project.
squawk
- Squawk – A Linter for Postgres Migrations
-
Postgres Language Server: Implementing the Parser
that is definitely the goal, both a formatter and a linter. we want to add something like squawk directly to the language server, so you get eslint-like dx. with the ast and the database schema in the data model, you can basically add any rule you like.
[0] https://github.com/sbdchd/squawk/tree/master
-
Bluesky migrates from pg to single-tenant SQLite
> I wish there were ways to enforce this on the db so you never accidentally grabbed a table lock during these operations.
You can use a linter for PostgreSQL migrations https://squawkhq.com/
-
PostgreSQL 14 Released
We've had good success with https://github.com/sbdchd/squawk to lint migrations. It tells you if a query is going to lock your table (as long as it's written in SQL, not some ORM DSL)
- GitHub has Degraded Availability
What are some alternatives?
ANTLR - ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.
zombodb - Making Postgres and Elasticsearch work together like it's 2023
H2 - H2 is an embeddable RDBMS written in Java.
gh-ost - GitHub's Online Schema-migration Tool for MySQL
grammars-v4 - Grammars written for ANTLR v4; expectation that the grammars are free of actions.
pgsanity - Check syntax of postgresql sql files
postbird - Open source PostgreSQL GUI client for macOS, Linux and Windows
pg_tm_aux - Transfer manager auxiliary functions
walrus - Applying RLS to PostgreSQL WAL
sqlx - 🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, and SQLite.
patroni - A template for PostgreSQL High Availability with Etcd, Consul, ZooKeeper, or Kubernetes
stolon - PostgreSQL cloud native High Availability and more.