Changing Tires at 100mph: A Guide to Zero Downtime Migrations

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • gh-ost

    GitHub's Online Schema-migration Tool for MySQL

    Actually I never tried but I was scared by the small print of GH not using RDS themselves [1] and Ghost relying on lower-level features that might be not easily available in RDS. Also I had the impression you have to setup a normal non-RDS replica attached to your RDS master?

    [1] https://github.com/github/gh-ost/blob/master/doc/rds.md

  • pg-osc

    Easy CLI tool for making zero downtime schema changes and backfills in PostgreSQL

    Postgres has some less robust tooling, at least both of these carry the "experimental" tag:

    https://news.ycombinator.com/item?id=29825520

    https://github.com/shayonj/pg-osc

    But, still, they do it for you.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • reshape

    An easy-to-use, zero-downtime schema migration tool for Postgres

    Anybody interested in this subject might also be interested in a tool for Postgres I’ve been working on, Reshape: https://github.com/fabianlindfors/reshape. It aims to fully automate away all the pain and manual steps zero-downtime migrations normally requires :)

  • database-lab-engine

    DBLab enables 🖖 database branching and ⚡️ thin cloning for any Postgres database and empowers DB testing in CI/CD. This optimizes database-related costs while improving time-to-market and software quality. Follow to stay updated.

  • litestream

    Streaming replication for SQLite.

    I imagine that GP means a hardware failure severe enough that it permanently borks the disk drive of the machine. In that case vanilla SQLite would indeed lose data written since the time of the last offsite backup. MySQL/Postgres/etc typically have streaming replication which means you can promote one of the replicas and not lose any (or at least much less) data. There are some new-ish projects like Litestream (https://litestream.io/) that provide a similar functionality for SQLite by streaming the WAL file changes to S3.

  • ghost_adapter

    Run ActiveRecord migrations through gh-ost

    At least for Rails there are several gems available (ie https://github.com/WeTransfer/ghost_adapter or https://github.com/departurerb/departure) that seamlessly hook into the existing migration system and will run all eligible migrations through gh-ost or pt-osc as needed. You're right that it's not free but it isn't all that far off either.

    That said, online schema migrations are a specialized tool designed for very big tables that take hours to run an ALTER TABLE on. If all your tables are small enough that alterations take less than a second or so, don't bother and just block the table for a bit. It's fine.

  • departure

    Percona's pt-online-schema-change runner for ActiveRecord migrations.

    At least for Rails there are several gems available (ie https://github.com/WeTransfer/ghost_adapter or https://github.com/departurerb/departure) that seamlessly hook into the existing migration system and will run all eligible migrations through gh-ost or pt-osc as needed. You're right that it's not free but it isn't all that far off either.

    That said, online schema migrations are a specialized tool designed for very big tables that take hours to run an ALTER TABLE on. If all your tables are small enough that alterations take less than a second or so, don't bother and just block the table for a bit. It's fine.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • janet

    A dynamic language and bytecode vm

    The "boots up our application within a second on a Raspberry Pi" comment in the FAQ[0] is actually about my development setup, as I don't own a fancy computer myself. It was mainly meant to illustrate the fast startup time of the Janet[1] language. In fact, that startup time is so fast, you can run your app as a CGI script. That has some more possible advantages, see the FAQ for more details.

    [0] FAQ: https://withoutdistractions.com/cv/faq#technology

    [1] Janet: https://janet-lang.org/

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts