Hacking the Postgres Wire Protocol

1. libpg_query

   1 19 1,331 8.0 C

   C library for accessing the PostgreSQL parser outside of the server environment

   

   Yes, the same way. It's all based on the extremely useful `https://github.com/pganalyze/libpg_query` project, which is where the "extracted the parser from Postgres" part comes in.

   Supabase's LSP also uses tree-sitter for corrections and autocomplete, because one drawback of using the server's source is that pg_query only works on well-formed/executable SQL - when it detects a malformed query, it formulates an error and exits, since that's what you want in an SQL server. So for partially-correct syntax and fill-forward, tree-sitter covers the gaps.

2. Stream

   getstream.io featured

   Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.

   

3. schemamap

   2 1 69 8.7 Rust

   Instant batch data import for Postgres

   

   > One of the things that puts metaprogramming on wheels is being able to pull in grammars, schemas and such for external systems.

   That's what I've been trying to do with: https://github.com/schemamap/schemamap

   For a well-constrained Postgres schema, a deterministic SQL compiler can be built (with plenty of logic programming/constraint-solving/hypergraph-analysis) that can integrate arbitrary external systems data.

   While this is NP-hard, with some clever use of embeddings and ontologies, and use of every single constraint AST within the DB to reduce the search space, this becomes feasible problem.

   For any Clojurists interested, I've packaged `pg_query`, so you can use it in your applications: https://github.com/schemamap/pg-query-clj

   At the moment I'm saving up for the next development cycle, so not only PG->PG schema mappings can be solved for (JSON schema is next!). Hope this sounds interesting :)

4. pg-query-clj

   3 1 9 2.4 Clojure

   Clojure library to parse, deparse and normalize SQL queries using the PostgreSQL query parser

   

   > One of the things that puts metaprogramming on wheels is being able to pull in grammars, schemas and such for external systems.

   That's what I've been trying to do with: https://github.com/schemamap/schemamap

   For a well-constrained Postgres schema, a deterministic SQL compiler can be built (with plenty of logic programming/constraint-solving/hypergraph-analysis) that can integrate arbitrary external systems data.

   While this is NP-hard, with some clever use of embeddings and ontologies, and use of every single constraint AST within the DB to reduce the search space, this becomes feasible problem.

   For any Clojurists interested, I've packaged `pg_query`, so you can use it in your applications: https://github.com/schemamap/pg-query-clj

   At the moment I'm saving up for the next development cycle, so not only PG->PG schema mappings can be solved for (JSON schema is next!). Hope this sounds interesting :)

5. onepassword-sdk-go

   4 3 83 8.8 Go

   

   Unless the system that wishes to consume tree-sitter grammars has access to a linker, it being written in C, and then compiled to machine code, gravely limits the places it can be consumed. That's in contrast to any one of the hundreds of grammar compiler compiler systems that allow targeting a platform of choice without mandating a platform of choice. I do see that tree-sitter alleges to emit wasm, but I am not deep enough in that ecosystem to know if it means "wasm for the browser," or "wasm for extism[1]," or "wasm, but if you already have WASI" or what

   1: e.g. https://github.com/1Password/onepassword-sdk-go/blob/v0.2.1/...

6. orafce

   5 4 510 8.2 C

   The "orafce" project implements in Postgres some of the functions from the Oracle database that are missing (or behaving differently).Those functions were verified on Oracle 10g, and the module is useful for production work.

   

   Is there even a grammar for Oracle SQL? I took a swing at trying to slurp it out of the "alt text" of the railroad images in the many, many html files in (e.g. https://docs.oracle.com/en/database/oracle/oracle-database/2... -> https://docs.oracle.com/en/database/oracle/oracle-database/2... ) but then I thankfully moved off any project from that point on that used Oracle and let it go

   I'm aware of https://github.com/orafce/orafce and https://github.com/pgoracle before that (the commit history also references ora2pg and searching for that surfaces https://github.com/darold/ora2pg which was just updated yesterday!) but I've never tried to use any of them in anger

7. ora2pg

   6 4 1,115 8.5 Perl

   Ora2Pg is a free tool used to migrate an Oracle database to a PostgreSQL compatible schema. It connects your Oracle database, scan it automatically and extracts its structure or data, it then generates SQL scripts that you can load into PostgreSQL.

   

   Is there even a grammar for Oracle SQL? I took a swing at trying to slurp it out of the "alt text" of the railroad images in the many, many html files in (e.g. https://docs.oracle.com/en/database/oracle/oracle-database/2... -> https://docs.oracle.com/en/database/oracle/oracle-database/2... ) but then I thankfully moved off any project from that point on that used Oracle and let it go

   I'm aware of https://github.com/orafce/orafce and https://github.com/pgoracle before that (the commit history also references ora2pg and searching for that surfaces https://github.com/darold/ora2pg which was just updated yesterday!) but I've never tried to use any of them in anger

8. vitess

   7 65 19,856 9.8 Go

   Vitess is a database clustering system for horizontal scaling of MySQL.

   

   > I am wondering, why it took so long for something like this to happen for Postgres

   > We were thinking about adding PostgreSQL support to Vitess several years ago. However, we realized that it would be more efficient if each tool was focused on a specific task.

   https://github.com/vitessio/vitess/issues/7084#issuecomment-...

9. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

Suggest a related project