refinery VS pggen

Great write up! At PeerDB, we’ve been using refinery https://github.com/rust-db/refinery to handle database migrations of our catalog Postgres database from Rust. It is easy, typesafe and gets the job done. Thought this would be useful for users building apps with Rust!

I don't think migrations need to be tied to ORM. We have refinery that allows you to write migrations in rust. I, personally, didn't like and prefer writing them in SQL - why learn how to migrations in X when you already know the SQL dialect that you're using.

The best solution I've ever seen is this Rust library https://github.com/cornucopia-rs/cornucopia

You write plain SQL for you schema (just a schema.sql is enough) and plain SQL functions for your queries. Then it generates Rust types and Rust functions from from that. If you don't use Rust, maybe there's a library like that for your favorite language.

Optionally, pair it with https://github.com/bikeshedder/tusker or https://github.com/blainehansen/postgres_migrator (both are based off https://github.com/djrobstep/migra) to generate migrations by diffing your schema.sql files, and https://github.com/rust-db/refinery to perform those migrations.

Now, if you have simple crud needs, you should probably use https://postgrest.org/en/stable/ and not an ORM. There are packages like https://www.npmjs.com/package/@supabase/postgrest-js (for JS / typescript) and probably for other languages too.

If you insist on an ORM, the best of the bunch is prisma https://www.prisma.io/ - outside of the typescript/javascript ecosystem it has ports for some other languages (with varying degrees of completion), the one I know about is the Rust one https://prisma.brendonovich.dev/introduction

What I personally use here is Refinery, but it's within the Rust ecosystem. It lets you write your migrations in pure SQL - which is what I prefer - but still requires about 5 lines of boilerplate Rust code.

Example refinery migrations

Cornucopia is neat. I wrote a similar library in Go [1] so I'm very interested in comparing design decisions.

The pros of the generated code per query approach:

- App code is coupled to query outputs and inputs (an API of sorts), not database tables. Therefore, you can refactor your DB without changing app code.

- Real SQL with the full breadth of DB features.

- Real type-checking with what the DB supports.

The cons:

- Type mapping is surprisingly hard to get right, especially with composite types and arrays and custom type converters. For example, a query might return multiple jsonb columns but the app code wants to parse them into different structs.

- Dynamic queries don't work with prepared statements. Prepared statements only support values, not identifiers or scalar SQL sub-queries, so the codegen layer needs a mechanism to template SQL. I haven't built this out yet but would like to.

[1]: https://github.com/jschaf/pggen

pggen is another fantastic library in this genre, which specifically targets postgres. It is driven by pgx. Can not recommend enough.

> Do you write out 120 "INSERT" statements, 120 "UPDATE" statements, 120 "DELETE" statements as raw strings

Yes. For example: https://github.com/jschaf/pggen/blob/main/example/erp/order/....

> that is also using an ORM

ORM as a term covers a wide swathe of usage. In the smallest definition, an ORM converts DB tuples to Go structs. In common usage, most folks use ORM to mean a generic query builder plus the type conversion from tuples to structs. For other usages, I prefer the Patterns of Enterprise Application Architecture terms [1] like data-mapper, active record, and table-data gateway.

[1]: https://martinfowler.com/eaaCatalog/

You might like pggen (I’m the author) which only supports Postgres and pgx. https://github.com/jschaf/pggen

pggen occupies the same design space as sqlc but the implementations are quite different. Sqlc figures out the query types using type inference in Go which is nice because you don’t need Postgres at build time. Pggen asks Postgres what the query types are which is nice because it works with any extensions and arbitrarily complex queries.

Any reason to use sqlc over pggen ? If you use Postgres, it seems like the superior option.

Agree with your choices, except go-json which I never tried. pggen is fantastic. Love that library. The underlying driver, pgx, is also really well written.

You might like the approach I took with pggen[1] which was inspired by sqlc[2]. You write a SQL query in regular SQL and the tool generates a type-safe Go querier struct with a method for each query.

The primary benefit of pggen and sqlc is that you don't need a different query model; it's just SQL and the tools automate the mapping between database rows and Go structs.

[1]: https://github.com/jschaf/pggen

[2]: https://github.com/kyleconroy/sqlc

I created pggen a few weeks ago to create my preferred method of database interaction: I write real SQL queries and I use generated, type-safe Go interfaces to the queries. https://github.com/jschaf/pggen