refinery VS languages-that-compile-to-sql

Great write up! At PeerDB, we’ve been using refinery https://github.com/rust-db/refinery to handle database migrations of our catalog Postgres database from Rust. It is easy, typesafe and gets the job done. Thought this would be useful for users building apps with Rust!

I don't think migrations need to be tied to ORM. We have refinery that allows you to write migrations in rust. I, personally, didn't like and prefer writing them in SQL - why learn how to migrations in X when you already know the SQL dialect that you're using.

The best solution I've ever seen is this Rust library https://github.com/cornucopia-rs/cornucopia

You write plain SQL for you schema (just a schema.sql is enough) and plain SQL functions for your queries. Then it generates Rust types and Rust functions from from that. If you don't use Rust, maybe there's a library like that for your favorite language.

Optionally, pair it with https://github.com/bikeshedder/tusker or https://github.com/blainehansen/postgres_migrator (both are based off https://github.com/djrobstep/migra) to generate migrations by diffing your schema.sql files, and https://github.com/rust-db/refinery to perform those migrations.

Now, if you have simple crud needs, you should probably use https://postgrest.org/en/stable/ and not an ORM. There are packages like https://www.npmjs.com/package/@supabase/postgrest-js (for JS / typescript) and probably for other languages too.

If you insist on an ORM, the best of the bunch is prisma https://www.prisma.io/ - outside of the typescript/javascript ecosystem it has ports for some other languages (with varying degrees of completion), the one I know about is the Rust one https://prisma.brendonovich.dev/introduction

What I personally use here is Refinery, but it's within the Rust ecosystem. It lets you write your migrations in pure SQL - which is what I prefer - but still requires about 5 lines of boilerplate Rust code.

Example refinery migrations

> Therefore, you can refactor your DB without changing app code.

Indeed, that's incredible! I tend to think that the actual layout of db tables a low level concern that is mainly driven by query performance and simplicity.

> - Type mapping is surprisingly hard to get right, especially with composite types and arrays and custom type converters. For example, a query might return multiple jsonb columns but the app code wants to parse them into different structs.

In Rust there's serde for converting json into structs (or enums if the shape of the json is complicated enough). Doesn't Go have a similar serialization/deserialization library?

> - Dynamic queries don't work with prepared statements. Prepared statements only support values, not identifiers or scalar SQL sub-queries, so the codegen layer needs a mechanism to template SQL. I haven't built this out yet but would like to.

In this case, what about stored procedures?

But, generally, instead of a template language for SQL, I would like to have a real compile-to-SQL higher level language, that could also do dynamic queries. The trouble with templating is that dynamic queries are hard to do in a type-safe manner, and it's hard to prevent generating invalid SQL when you have a bad template substitution (and then you get bad error messages from the db). There's a few languages like this do this https://github.com/ajnsit/languages-that-compile-to-sql but none fits the bill