pg-osc
postgres_migrator
pg-osc | postgres_migrator | |
---|---|---|
7 | 7 | |
486 | 82 | |
- | - | |
7.6 | 5.1 | |
about 1 month ago | 4 months ago | |
Ruby | Rust | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pg-osc
-
How not to change PostgreSQL column type
My thoughts exactly. It's surprising that external online schema change tools for Postgres have only become a thing fairly recently! The only two I'm aware of are:
* pgroll: Written in Golang, first commits June 2023. https://github.com/xataio/pgroll
* pg-osc: Written in Ruby, first commits Dec 2021. https://github.com/shayonj/pg-osc
Meanwhile over in the MySQL and MariaDB ecosystem, external OSC tools have been around for quite some time, starting with oak-online-alter-table over 15 years ago. The most popular options today are pt-online-schema-change or gh-ost, but other battle-tested solutions include fb-osc, LHM, and the latest entry Spirit.
-
Pgroll: zero-downtime, undoable, schema migrations for Postgres
Great to see more innovation in this space! How does this compare to?
https://github.com/shayonj/pg-osc
- Want to avoid MySQL but find PlanetScale really appealing
-
Changing Tires at 100mph: A Guide to Zero Downtime Migrations
Postgres has some less robust tooling, at least both of these carry the "experimental" tag:
https://news.ycombinator.com/item?id=29825520
https://github.com/shayonj/pg-osc
But, still, they do it for you.
- Alter PostgreSQL tables without locks and downtime
-
pg-osc: Zero downtime non blocking schema changes in PostgreSQL
You can also try it out, Github: https://github.com/shayonj/pg-osc
-
Show HN: Zero-downtime PostgreSQL migrations for Ruby on Rails
This is very cool! I no longer work on Rails on daily basis, but would have loved to have this.
I built something similar called `pg-osc` / `pg-online-schema-change` for PostgreSQL workloads. It does zero downtime migrations using triggers and shadow table.
Github: https://github.com/shayonj/pg-osc.
Blog: https://www.shayon.dev/post/2022/47/pg-osc-zero-downtime-sch...
Also love that this is written in Ruby :). Going to check it out on some side projects.
postgres_migrator
-
We built our customer data warehouse all on Postgres
Thanks! Yeah definitely agree that building out declarative table management for Postgres would be a major effort. A few open source projects I've seen in that area include:
https://github.com/sqldef/sqldef (Go)
https://github.com/bikeshedder/tusker (Python but being ported to Rust)
https://github.com/tyrchen/renovate (Rust)
https://github.com/blainehansen/postgres_migrator (Rust)
Some of these are based on parsing SQL, and others are based on running the CREATEs in a temporary location and introspecting the result.
The schema export side can be especially tricky for Postgres, since it lacks a built-in equivalent to MySQL's SHOW CREATE TABLE. So most of these declarative pg tools shell out to pg_dump, or require the user to do so. But sqldef actually implements CREATE TABLE dumping in pure Golang if I recall correctly, which is pretty cool.
There's also the question of implementing the table diff logic from scratch, vs shelling out to another tool or using a library. For the latter path, there's a nice blog post from Supabase about how they evaluated the various options: https://supabase.com/blog/supabase-cli#choosing-the-best-dif...
-
Pgroll: zero-downtime, undoable, schema migrations for Postgres
Forr postgres, how does the schema diffing aspect compare to migra?
https://github.com/djrobstep/migra
I'm asking because, although migra is excellent and there are multiple migrations tools based on it (at least https://github.com/bikeshedder/tusker and https://github.com/blainehansen/postgres_migrator), issues are piling up but development seem to be slowing down
-
Diesel 2.1
Is this similar to migra? There's a tool written in Rust that calls it, postgres_migrator (there's also tusker)
-
Ask HN: ORM or Native SQL?
The best solution I've ever seen is this Rust library https://github.com/cornucopia-rs/cornucopia
You write plain SQL for you schema (just a schema.sql is enough) and plain SQL functions for your queries. Then it generates Rust types and Rust functions from from that. If you don't use Rust, maybe there's a library like that for your favorite language.
Optionally, pair it with https://github.com/bikeshedder/tusker or https://github.com/blainehansen/postgres_migrator (both are based off https://github.com/djrobstep/migra) to generate migrations by diffing your schema.sql files, and https://github.com/rust-db/refinery to perform those migrations.
Now, if you have simple crud needs, you should probably use https://postgrest.org/en/stable/ and not an ORM. There are packages like https://www.npmjs.com/package/@supabase/postgrest-js (for JS / typescript) and probably for other languages too.
If you insist on an ORM, the best of the bunch is prisma https://www.prisma.io/ - outside of the typescript/javascript ecosystem it has ports for some other languages (with varying degrees of completion), the one I know about is the Rust one https://prisma.brendonovich.dev/introduction
- migrator: automatically generate postgres migrations from a declarative raw sql schema
- migrator: automatically generate postgres migrations from a declarative raw sql schema, written in Rust
- Migrator: Automatically generate Postgres migrations from declarative SQL schema
What are some alternatives?
pg-online-schema-change - Easy CLI tool for making zero downtime schema changes and backfills in PostgreSQL [Moved to: https://github.com/shayonj/pg-osc]
refinery - Powerful SQL migration toolkit for Rust.
safe-pg-migrations - Make your PostgreSQL migrations safe
pgroll - PostgreSQL zero-downtime migrations made easy
reshape - An easy-to-use, zero-downtime schema migration tool for Postgres
migra - Like diff but for PostgreSQL schemas
django-pg-zero-downtime-migrations - Django postgresql backend that apply migrations with respect to database locks
quantumdb - Zero-downtime schema evolution for PostgreSQL
tusker - PostgreSQL migration management tool
safeql - Composable / async / functional / type-safe / parallel-pipelined queries and relations without SQL injection or N+1s.