How Modern SQL Databases Are Changing Web Development - #3 Better Developer Experience

• gh-ost

  32 11,997 7.4 Go

  GitHub's Online Schema-migration Tool for MySQL

  

I’ve been through multiple incidents where everything worked fine in the testing environment but ended up locking the production database for minutes when deployed. A category of open-source tools called OSC (Online Schema Change) exists to mitigate such pain, like gh-ost used by GitHub and OSC used by Meta. They work by creating a set of "ghost tables" to apply the migrations, copy over old data from the original tables, and catch up with new writes simultaneously. When all old data is migrated, you can trigger a cutover to make the "ghost tables" production. Check the post below for a great introduction and comparison:

• copycat

  13 314 8.8 TypeScript

  Generate deterministic fake values: The same input will always generate the same fake-output. (by snaplet)

  

It’s worth noting that being able to branch production data for testing easily doesn’t mean you should just do it. It poses a significant risk of leaking sensitive user data. You should consider using tools like Snaplet to transform and anonymize sensitive columns.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• OnlineSchemaChange

  3 955 8.1 Python

  A tool for performing online schema changes on MySQL.

  

I’ve been through multiple incidents where everything worked fine in the testing environment but ended up locking the production database for minutes when deployed. A category of open-source tools called OSC (Online Schema Change) exists to mitigate such pain, like gh-ost used by GitHub and OSC used by Meta. They work by creating a set of "ghost tables" to apply the migrations, copy over old data from the original tables, and catch up with new writes simultaneously. When all old data is migrated, you can trigger a cutover to make the "ghost tables" production. Check the post below for a great introduction and comparison:

• neon

  123 12,279 9.9 Rust

  Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, branching, and bottomless storage.

  

I like how Neon put it in simple words:

Suggest a related project