reshape
strong_migrations
Our great sponsors
reshape | strong_migrations | |
---|---|---|
16 | 16 | |
1,598 | 3,808 | |
- | - | |
7.4 | 8.1 | |
11 days ago | 17 days ago | |
Rust | Ruby | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
reshape
-
Pgroll: zero-downtime, undoable, schema migrations for Postgres
Cool stuff! Do you have any thoughts about how this compares to https://github.com/fabianlindfors/reshape?
-
Postgres schema changes are still a PITA
From what I know, there is only one project that tries something close to this: the relatively recent Reshape. It uses Postgres views to expose the two versions of the schema and triggers to upgrade/downgrade the new data. It doesn’t do the constraints part as described above, but shows that this approach is possible. Combined with the Xata pull request based workflow, I think the ideal system described above is possible!
-
Conceptually how do you handle deploys of SQL related things (table definition, scripts, stored procs etc) in a CI/CD way?
My idea is not unique. Reshape is similar, but bigger in scope.
Shameless plug: I’ve been working on a tool called Reshape in this space: https://github.com/fabianlindfors/reshape. It fully automates zero-downtime schema migrations and avoids all the manual work that has been discussed in the comments here.
-
Changing Tires at 100mph: A Guide to Zero Downtime Migrations
Anybody interested in this subject might also be interested in a tool for Postgres I’ve been working on, Reshape: https://github.com/fabianlindfors/reshape. It aims to fully automate away all the pain and manual steps zero-downtime migrations normally requires :)
-
PostgreSQL at Scale: Database Schema Changes Without Downtime
This post is absolutely terrific and has been been my main reference for Reshape, an automated, zero-downtime schema migration tool: https://github.com/fabianlindfors/reshape
- When Postgres blocks: tips for dealing with locks
-
Atlas – Terraform but for Database Migrations
I thought the established wisdom is to make schema migration compatible with both the old app and the new app, whenever it is possible? E.g. you can safely add a nullable column, and it shouldn't trip up the old app nor the new app, unless you are using some crappy ORM that does "SELECT *", or you are on an older MySQL version that may take hours/days/weeks to rewrite the whole table just to add a nullable column.
https://github.com/fabianlindfors/reshape, on HN front page a couple weeks ago, has some nice tricks to help with the incompatible cases.
-
Why I Enjoy PostgreSQL – Infrastructure Engineer's Perspective
Great article! For anybody interested in this topic, I've been working on a schema migration tool which automates zero-downtime migrations using many of the techniques mentioned: https://github.com/fabianlindfors/reshape. It also uses some other incredible Postgres features, like updatable views and schemas.
It was discussed here on HN about a week back: https://news.ycombinator.com/item?id=29825520
-
Zero-downtime schema migrations in Postgres using Reshape
> We have migrated a few fields from varchar to integer. How would your solution deal with this
There is actually an example in the README on how to change a column from TEXT to INTEGER, the technique would be the same the other way around: https://github.com/fabianlindfors/reshape#alter-column
For the cases that require manual handling, that is a bit tricky. I'm not sure Reshape would be able to automate that in any meaningful way so the best thing might just be to fall back to a standard procedure of making the changes in two separate migrations, where the manual changes are done in between.
> Another one is adding foreign keys where the existing data does not conform to the foreign key constraint.
I have given this some thought before as I wanted to add a migration which can add new foreign keys. I think it can be done by writing some migrations first which update the existing data to conform to the constraint, for example adding missing values. This can be done with `alter_column` right now and there will be more comprehensive migrations for data changes in the future.
strong_migrations
-
Must-have gems for mature Rails
gem "strong_migrations" - https://github.com/ankane/strong_migrations | Helps devs write non-blocking migrations, a must-have.
-
How does Rails handle out of order migrations (when working on different local branches)
There’s no real way to test, but you can use gems like https://github.com/ankane/strong_migrations and not allow to merge branches unless they are up-to-date with main.
-
When Postgres blocks: tips for dealing with locks
Half of the problems in this article are migration related.
I am extremely grateful that some people have created awesome libraries like strong migrations https://github.com/ankane/strong_migrations. Even if you are not using rails, bookmark its readme, it is an awesome cheat-sheet when writing a migration.
- Best practices as code using RuboCop
-
Why I Enjoy PostgreSQL – Infrastructure Engineer's Perspective
I would suggest taking a look at strong migrations[1]. It's a rails project, but the readme does a great job explaining what it checks for and what safe alternative to use instead. I still link to their explanations in PRs for non-rails projects.
-
Database... or Goose?
Nice one! Personally, I've been using https://github.com/ankane/strong_migrations to cover this case.
-
Ten Ruby gems for Rails you should definitely know about
StrongMigrations
-
Elixir and Phoenix after two years
> Ecto prevents N+1 queries by default, which I think is clearly better.
To be fair...
If you want to protect yourself from these with Rails you can install Bullet[0] and get protection through in your face notifications, and you have the option to let it slide because you're taking advantage of caching with Rails and in this case you know what you're getting into and the N+1 query with caching ends up being better because you understand your domain.
Rails also has the strong migrations[1] gem which is a huge help for not shooting yourself in the foot for running migrations in production by helping you avoid table locks and other issues / errors. But AFAIK there's no Ecto equivalent, but strong migrations is really really useful.
Rails also has the data-migrate gem which is a nice little abstraction for splitting out your schema changes and backfilling data in an automated way. There's nothing like with Ecto. This one isn't as useful as strong migrations IMO but it's still very handy to have this problem taken care of for you without having to re-invent a new strategy in every project or copy code over.
Basically all 3 of these things are something I'd use in every project in Rails but with Phoenix I wouldn't have these things except for N+1 query protection.
[0]: https://github.com/flyerhzm/bullet
What are some alternatives?
safe-pg-migrations - Make your PostgreSQL migrations safe
money-rails - Integration of RubyMoney - Money with Rails
phony_rails - This Gem adds useful methods to your Rails app to validate, display and save phone numbers. It uses the super awesome Phony gem (https://github.com/floere/phony).
pg-online-schema-change - Easy CLI tool for making zero downtime schema changes and backfills in PostgreSQL [Moved to: https://github.com/shayonj/pg-osc]
pgroll - PostgreSQL zero-downtime migrations made easy
lockbox - Modern encryption for Ruby and Rails
data-migrate - Migrate and update data alongside your database structure.
migra - Like diff but for PostgreSQL schemas
Pagy - 🏆 The Best Pagination Ruby Gem 🥇
gh-ost - GitHub's Online Schema-migration Tool for MySQL
ghost_adapter - Run ActiveRecord migrations through gh-ost
atlas - A modern tool for managing database schemas