OnlineSchemaChange VS copycat

I’ve been through multiple incidents where everything worked fine in the testing environment but ended up locking the production database for minutes when deployed. A category of open-source tools called OSC (Online Schema Change) exists to mitigate such pain, like gh-ost used by GitHub and OSC used by Meta. They work by creating a set of "ghost tables" to apply the migrations, copy over old data from the original tables, and catch up with new writes simultaneously. When all old data is migrated, you can trigger a cutover to make the "ghost tables" production. Check the post below for a great introduction and comparison:

For larger companies, with more data and high-traffic tables, schema changes may happen less often, but they still need to worry about things like downtime caused by locking. They require long internal guides on performing schema changes correctly (e.g. GitLab, PayPal), custom tools (e.g. Meta, Square), and they often document incidents or near-incidents caused by schema migrations (e.g. GitHub, Doctolib, GoCardless).

Check out https://www.snaplet.dev (I'm the founder). We do exactly this.

Testing with product workloads today? If you need to test with production workloads today, check out Snaplet and Postgres.ai. Both are great partners of Supabase.

It’s worth noting that being able to branch production data for testing easily doesn’t mean you should just do it. It poses a significant risk of leaking sensitive user data. You should consider using tools like Snaplet to transform and anonymize sensitive columns.

[disclosure: I'm the founder of Snaplet]

I think there are a lot of different reasons why people may want to use a service like bit.io, but if you want a database with data in it to code against, run tests against, reproduce production related data-bugs, and run e2e tests against then check out https://www.snaplet.dev.

For now, this is very experimental - but it has a lot of potential. If you want to get involved, please reach out to us or the team at [Snaplet(https://www.snaplet.dev/). The work they're doing over at Snaplet is incredible, and we've had a blast collaborating with them.