reshape
postgres_migrator
reshape | postgres_migrator | |
---|---|---|
16 | 7 | |
1,653 | 78 | |
- | - | |
7.4 | 5.1 | |
about 2 months ago | 3 months ago | |
Rust | Rust | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
reshape
- Show HN: Reshape – Zero-downtime schema migrations for Postgres
-
Pgroll: zero-downtime, undoable, schema migrations for Postgres
Cool stuff! Do you have any thoughts about how this compares to https://github.com/fabianlindfors/reshape?
-
Postgres schema changes are still a PITA
From what I know, there is only one project that tries something close to this: the relatively recent Reshape. It uses Postgres views to expose the two versions of the schema and triggers to upgrade/downgrade the new data. It doesn’t do the constraints part as described above, but shows that this approach is possible. Combined with the Xata pull request based workflow, I think the ideal system described above is possible!
-
Conceptually how do you handle deploys of SQL related things (table definition, scripts, stored procs etc) in a CI/CD way?
My idea is not unique. Reshape is similar, but bigger in scope.
-
The operational relational schema paradigm
This is precisely what I did for my automated zero-downtime migration tool and it works pretty well: https://github.com/fabianlindfors/reshape. At least for Postgres, simple views like these have almost no overhead as queries are simply rewritten for the underlying table.
-
Changing Tires at 100mph: A Guide to Zero Downtime Migrations
Anybody interested in this subject might also be interested in a tool for Postgres I’ve been working on, Reshape: https://github.com/fabianlindfors/reshape. It aims to fully automate away all the pain and manual steps zero-downtime migrations normally requires :)
- Reshape: An experimental, easy-to-use, zero-downtime migration tool for Postgres
-
PostgreSQL at Scale: Database Schema Changes Without Downtime
This post is absolutely terrific and has been been my main reference for Reshape, an automated, zero-downtime schema migration tool: https://github.com/fabianlindfors/reshape
- When Postgres blocks: tips for dealing with locks
- Reshape
postgres_migrator
-
We built our customer data warehouse all on Postgres
Thanks! Yeah definitely agree that building out declarative table management for Postgres would be a major effort. A few open source projects I've seen in that area include:
https://github.com/sqldef/sqldef (Go)
https://github.com/bikeshedder/tusker (Python but being ported to Rust)
https://github.com/tyrchen/renovate (Rust)
https://github.com/blainehansen/postgres_migrator (Rust)
Some of these are based on parsing SQL, and others are based on running the CREATEs in a temporary location and introspecting the result.
The schema export side can be especially tricky for Postgres, since it lacks a built-in equivalent to MySQL's SHOW CREATE TABLE. So most of these declarative pg tools shell out to pg_dump, or require the user to do so. But sqldef actually implements CREATE TABLE dumping in pure Golang if I recall correctly, which is pretty cool.
There's also the question of implementing the table diff logic from scratch, vs shelling out to another tool or using a library. For the latter path, there's a nice blog post from Supabase about how they evaluated the various options: https://supabase.com/blog/supabase-cli#choosing-the-best-dif...
-
Pgroll: zero-downtime, undoable, schema migrations for Postgres
Forr postgres, how does the schema diffing aspect compare to migra?
https://github.com/djrobstep/migra
I'm asking because, although migra is excellent and there are multiple migrations tools based on it (at least https://github.com/bikeshedder/tusker and https://github.com/blainehansen/postgres_migrator), issues are piling up but development seem to be slowing down
-
Diesel 2.1
Is this similar to migra? There's a tool written in Rust that calls it, postgres_migrator (there's also tusker)
-
Ask HN: ORM or Native SQL?
The best solution I've ever seen is this Rust library https://github.com/cornucopia-rs/cornucopia
You write plain SQL for you schema (just a schema.sql is enough) and plain SQL functions for your queries. Then it generates Rust types and Rust functions from from that. If you don't use Rust, maybe there's a library like that for your favorite language.
Optionally, pair it with https://github.com/bikeshedder/tusker or https://github.com/blainehansen/postgres_migrator (both are based off https://github.com/djrobstep/migra) to generate migrations by diffing your schema.sql files, and https://github.com/rust-db/refinery to perform those migrations.
Now, if you have simple crud needs, you should probably use https://postgrest.org/en/stable/ and not an ORM. There are packages like https://www.npmjs.com/package/@supabase/postgrest-js (for JS / typescript) and probably for other languages too.
If you insist on an ORM, the best of the bunch is prisma https://www.prisma.io/ - outside of the typescript/javascript ecosystem it has ports for some other languages (with varying degrees of completion), the one I know about is the Rust one https://prisma.brendonovich.dev/introduction
- migrator: automatically generate postgres migrations from a declarative raw sql schema
- migrator: automatically generate postgres migrations from a declarative raw sql schema, written in Rust
- Migrator: Automatically generate Postgres migrations from declarative SQL schema
What are some alternatives?
pg-online-schema-change - Easy CLI tool for making zero downtime schema changes and backfills in PostgreSQL [Moved to: https://github.com/shayonj/pg-osc]
refinery - Powerful SQL migration toolkit for Rust.
pgroll - PostgreSQL zero-downtime migrations made easy
migra - Like diff but for PostgreSQL schemas
gh-ost - GitHub's Online Schema-migration Tool for MySQL
quantumdb - Zero-downtime schema evolution for PostgreSQL
atlas - Manage your database schema as code
safeql - Composable / async / functional / type-safe / parallel-pipelined queries and relations without SQL injection or N+1s.
bytebase.com - Source for bytebase.com
tusker - PostgreSQL migration management tool