SaaSHub helps you find the best software and product alternatives Learn more →
Top 3 schema-management Open-Source Projects
-
Project mention: We built our customer data warehouse all on Postgres | news.ycombinator.com | 2024-02-02
Thanks! Yeah definitely agree that building out declarative table management for Postgres would be a major effort. A few open source projects I've seen in that area include:
https://github.com/sqldef/sqldef (Go)
https://github.com/bikeshedder/tusker (Python but being ported to Rust)
https://github.com/tyrchen/renovate (Rust)
https://github.com/blainehansen/postgres_migrator (Rust)
Some of these are based on parsing SQL, and others are based on running the CREATEs in a temporary location and introspecting the result.
The schema export side can be especially tricky for Postgres, since it lacks a built-in equivalent to MySQL's SHOW CREATE TABLE. So most of these declarative pg tools shell out to pg_dump, or require the user to do so. But sqldef actually implements CREATE TABLE dumping in pure Golang if I recall correctly, which is pretty cool.
There's also the question of implementing the table diff logic from scratch, vs shelling out to another tool or using a library. For the latter path, there's a nice blog post from Supabase about how they evaluated the various options: https://supabase.com/blog/supabase-cli#choosing-the-best-dif...
-
Project mention: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments? | news.ycombinator.com | 2024-03-27
I’d argue the obvious answer is address the lack of great answers for declarative schema migration in PostgreSQL. There is Skeema https://github.com/skeema/skeema but it doesn’t support Postgres and Prisma iirc forces you into an ORM, atlas looks perfect but has a nonstandard license.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Project mention: ORMs are nice but they are the wrong abstraction | news.ycombinator.com | 2024-02-01
The most interesting/fresh approach I've seen to this problem is Permazen.
https://github.com/permazen/permazen/
It starts by asking what the most natural way is to integrate persistence with a programming language (Java in this case, but the concepts are generic), and then goes ahead and implements the features of an RDBMS as an in-process library that can be given different storage backends as long as they implement a sorted K/V store. So it can sit on top of a simple in-process file based K/V store, RocksDB, FoundationDB, or any SQL database like PostgreSQL, SQLite, Spanner, etc (it just uses the RDBMS to store sorted key/value pairs in that case).
Essentially it's a way to map object graphs to key/value pairs but with the usual features you'd want like indexing, validation, transactions, and so on. The design is really nice and can scale from tiny tasks you'd normally use JSON or object serialization for, all the way up to large distributed clusters.
Because the native object model is mapped directly to storage there's no object/relational mismatch.
schema-management related posts
- How Meta Built the Infrastructure for Threads
- Automagically generate migrations for GORM
- AWS open source news and updates, #116
- Sqldef: Idempotent Schema Management for MySQL, PostgreSQL, SQLite, SQL Server
- Sqldef: Idempotent schema management for MySQL, PostgreSQL, SQLite, and more
-
A note from our sponsor - SaaSHub
www.saashub.com | 28 Mar 2024
Index
What are some of the best open-source schema-management projects? This list will help you:
Project | Stars | |
---|---|---|
1 | sqldef | 1,778 |
2 | skeema | 1,220 |
3 | Permazen | 397 |