reshape
pg-osc
Our great sponsors
reshape | pg-osc | |
---|---|---|
16 | 6 | |
1,617 | 475 | |
- | - | |
7.4 | 7.9 | |
about 1 month ago | 3 days ago | |
Rust | Ruby | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
reshape
-
Pgroll: zero-downtime, undoable, schema migrations for Postgres
Cool stuff! Do you have any thoughts about how this compares to https://github.com/fabianlindfors/reshape?
-
Postgres schema changes are still a PITA
From what I know, there is only one project that tries something close to this: the relatively recent Reshape. It uses Postgres views to expose the two versions of the schema and triggers to upgrade/downgrade the new data. It doesn’t do the constraints part as described above, but shows that this approach is possible. Combined with the Xata pull request based workflow, I think the ideal system described above is possible!
-
Conceptually how do you handle deploys of SQL related things (table definition, scripts, stored procs etc) in a CI/CD way?
My idea is not unique. Reshape is similar, but bigger in scope.
Shameless plug: I’ve been working on a tool called Reshape in this space: https://github.com/fabianlindfors/reshape. It fully automates zero-downtime schema migrations and avoids all the manual work that has been discussed in the comments here.
-
Changing Tires at 100mph: A Guide to Zero Downtime Migrations
Anybody interested in this subject might also be interested in a tool for Postgres I’ve been working on, Reshape: https://github.com/fabianlindfors/reshape. It aims to fully automate away all the pain and manual steps zero-downtime migrations normally requires :)
-
PostgreSQL at Scale: Database Schema Changes Without Downtime
This post is absolutely terrific and has been been my main reference for Reshape, an automated, zero-downtime schema migration tool: https://github.com/fabianlindfors/reshape
- When Postgres blocks: tips for dealing with locks
-
Atlas – Terraform but for Database Migrations
I thought the established wisdom is to make schema migration compatible with both the old app and the new app, whenever it is possible? E.g. you can safely add a nullable column, and it shouldn't trip up the old app nor the new app, unless you are using some crappy ORM that does "SELECT *", or you are on an older MySQL version that may take hours/days/weeks to rewrite the whole table just to add a nullable column.
https://github.com/fabianlindfors/reshape, on HN front page a couple weeks ago, has some nice tricks to help with the incompatible cases.
-
Why I Enjoy PostgreSQL – Infrastructure Engineer's Perspective
Great article! For anybody interested in this topic, I've been working on a schema migration tool which automates zero-downtime migrations using many of the techniques mentioned: https://github.com/fabianlindfors/reshape. It also uses some other incredible Postgres features, like updatable views and schemas.
It was discussed here on HN about a week back: https://news.ycombinator.com/item?id=29825520
-
Zero-downtime schema migrations in Postgres using Reshape
> We have migrated a few fields from varchar to integer. How would your solution deal with this
There is actually an example in the README on how to change a column from TEXT to INTEGER, the technique would be the same the other way around: https://github.com/fabianlindfors/reshape#alter-column
For the cases that require manual handling, that is a bit tricky. I'm not sure Reshape would be able to automate that in any meaningful way so the best thing might just be to fall back to a standard procedure of making the changes in two separate migrations, where the manual changes are done in between.
> Another one is adding foreign keys where the existing data does not conform to the foreign key constraint.
I have given this some thought before as I wanted to add a migration which can add new foreign keys. I think it can be done by writing some migrations first which update the existing data to conform to the constraint, for example adding missing values. This can be done with `alter_column` right now and there will be more comprehensive migrations for data changes in the future.
pg-osc
-
Pgroll: zero-downtime, undoable, schema migrations for Postgres
Great to see more innovation in this space! How does this compare to?
- Want to avoid MySQL but find PlanetScale really appealing
-
Changing Tires at 100mph: A Guide to Zero Downtime Migrations
Postgres has some less robust tooling, at least both of these carry the "experimental" tag:
https://news.ycombinator.com/item?id=29825520
https://github.com/shayonj/pg-osc
But, still, they do it for you.
-
pg-osc: Zero downtime non blocking schema changes in PostgreSQL
You can also try it out, Github: https://github.com/shayonj/pg-osc
-
Show HN: Zero-downtime PostgreSQL migrations for Ruby on Rails
This is very cool! I no longer work on Rails on daily basis, but would have loved to have this.
I built something similar called `pg-osc` / `pg-online-schema-change` for PostgreSQL workloads. It does zero downtime migrations using triggers and shadow table.
Github: https://github.com/shayonj/pg-osc.
Blog: https://www.shayon.dev/post/2022/47/pg-osc-zero-downtime-sch...
Also love that this is written in Ruby :). Going to check it out on some side projects.
What are some alternatives?
pg-online-schema-change - Easy CLI tool for making zero downtime schema changes and backfills in PostgreSQL [Moved to: https://github.com/shayonj/pg-osc]
pgroll - PostgreSQL zero-downtime migrations made easy
migra - Like diff but for PostgreSQL schemas
gh-ost - GitHub's Online Schema-migration Tool for MySQL
atlas - A modern tool for managing database schemas
safe-pg-migrations - Make your PostgreSQL migrations safe
bytebase.com - Source for bytebase.com
departure - Percona's pt-online-schema-change runner for ActiveRecord migrations.
Hasura - Blazing fast, instant realtime GraphQL APIs on your DB with fine grained access control, also trigger webhooks on database events.
strong_migrations - Catch unsafe migrations in development
sqlc - Generate type-safe code from SQL