sqldef
xo
sqldef | xo | |
---|---|---|
9 | 16 | |
1,815 | 3,567 | |
1.5% | 0.6% | |
9.3 | 6.2 | |
5 days ago | 10 days ago | |
Go | Go | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sqldef
-
We built our customer data warehouse all on Postgres
Thanks! Yeah definitely agree that building out declarative table management for Postgres would be a major effort. A few open source projects I've seen in that area include:
https://github.com/sqldef/sqldef (Go)
https://github.com/bikeshedder/tusker (Python but being ported to Rust)
https://github.com/tyrchen/renovate (Rust)
https://github.com/blainehansen/postgres_migrator (Rust)
Some of these are based on parsing SQL, and others are based on running the CREATEs in a temporary location and introspecting the result.
The schema export side can be especially tricky for Postgres, since it lacks a built-in equivalent to MySQL's SHOW CREATE TABLE. So most of these declarative pg tools shell out to pg_dump, or require the user to do so. But sqldef actually implements CREATE TABLE dumping in pure Golang if I recall correctly, which is pretty cool.
There's also the question of implementing the table diff logic from scratch, vs shelling out to another tool or using a library. For the latter path, there's a nice blog post from Supabase about how they evaluated the various options: https://supabase.com/blog/supabase-cli#choosing-the-best-dif...
-
Prisma laying off 28% staff
If you wish to auto-generate migrations, there are declarative schema change tools available for most relational databases. I'm the creator of Skeema [1] which provides them for MySQL, but there are options for other DBs too [2][3][4].
Prisma's migration system actually partially copied Skeema's design, while giving credit in a rather odd fashion which really rubbed me the wrong way: "The workflow of working with temporary databases and introspecting it to determine differences between schemas seems to be pretty common, this is for example what skeema does." [5]
While I doubt I was the first person to ever use that technique, I absolutely didn't copy it from anywhere, and it was never "pretty common". I'm not aware of any other older schema change systems that work this way.
[1] https://www.skeema.io
[2] https://github.com/djrobstep/migra
[3] https://github.com/k0kubun/sqldef
[4] https://david.rothlis.net/declarative-schema-migration-for-s...
[5] https://github.com/prisma/prisma-engines/blob/6be410e/migrat...
-
Allow Cloudflare D1 to be used like an O/R mapper with type-safe
By the way, I have also made a repository that uses sqldef as a migration tool to make migration as easy as possible, so please take a look at it.
-
AWS open source news and updates, #116
sqldef-gitops-cdk sqldef is an open source idempotent schema management for MySQL, PostgreSQL, and others. This repo provides a GitOps approach using AWS CDK of how you can approach database table migration.
- Sqldef: Idempotent Schema Management for MySQL, PostgreSQL, SQLite, SQL Server
- Sqldef: Idempotent schema management for MySQL, PostgreSQL, SQLite, and more
-
Migra: Like Diff but for PostgreSQL Schemas
I work heavily in this space and can add some more details :)
Tusker actually uses migra to power its functionality: https://github.com/bikeshedder/tusker#how-does-it-work
Tusker's flow is somewhat similar to sqldef https://github.com/k0kubun/sqldef , although the internal mechanics are quite different. Migra (and therefore Tusker) executes the SQL, introspects it, and diffs the introspected in-memory representation -- in other words, using the database directly as the canonical parser. In contrast, sqldef parses the SQL itself, builds an in-memory representation based on that, and then does the diff that way.
I'm the author of Skeema https://www.skeema.io which provides a similar declarative workflow for MySQL and MariaDB schema changes. Skeema uses an execute-and-introspect approach similar to Migra/Tusker, although each object is split out into its own .sql file for easier management in version control, with a multi-level directory hierarchy if you have multiple database instances and multiple schemas.
Skeema was conceptually inspired by Facebook's internal database schema change flow, as FB has used declarative schema management submission/review/execution company-wide for over a decade now. Skeema actually predates both Migra and sqldef slightly, although it did not influence them, all were developed separately.
In turn, Prisma Migrate and Vitess/PlanetScale declarative migrations were directly inspired by Skeema's approach, paradigms, and/or even direct use of source code in Vitess's case. (Although they're finally moving to a parser-based approach instead, which I recommended they do over a year ago, as it makes more sense for their use-case -- their whole product inherently requires a thorough SQL parser anyway...)
-
MariaDB to go public at $672M valuation
Thanks! I know of a couple Postgres tools that work in a declarative fashion: migra [1] and sqldef [2].
Migra is Postgres-specific. Its model is similar to Skeema's, in that the desired-state CREATEs are run in a temporary location and then introspected, to build an in-memory understanding of the desired state which can be diff'ed against the current actual state. (This approach was also borrowed by Prisma Migrate [3]). In this manner, the tool doesn't need a SQL parser, instead relying on the real DBMS to guarantee the CREATE is interpreted correctly with your exact DBMS version/flavor/settings.
In contrast, sqldef supports multiple databases, including Postgres and MySQL (among others). Unlike other tools, it uses a SQL parser-based approach to build its in-memory understanding of the desired state. As a DB professional, personally this approach scares me a bit, given the amount of nonstandard stuff in each DBMS's SQL dialect. But I'm inherently biased on this topic. And I will note sqldef's author is a core Ruby committer and JIT author, and is extremely skilled at parsers.
[1] https://databaseci.com/docs/migra
[2] https://github.com/k0kubun/sqldef
[3] https://github.com/prisma/prisma-engines/blob/main/migration...
-
Why Google Treats SQL Like Code and You Should Too
Declarative schema management tools make this much easier. The concept is your schema repo just stores CREATE statements, and the schema management tool knows how to generate DDL to transition between the current state in your DB and the desired state in your repo.
I'm the author of declarative schema management tool skeema (https://www.skeema.io, for MySQL / MariaDB). Some other options in this space are sqldef (https://github.com/k0kubun/sqldef, for MySQL or Postgres) and migra (https://github.com/djrobstep/migra, for Postgres). In MS SQL Server, SSDT DACPACs are also somewhat similar.
xo
-
Open-sourcing SQX, a way to build flexible database models in Go
i like xo's approach https://github.com/xo/xo but it is as is. I would love if something similar comes along that is used by db practititoners that is actively used and supported.
- PHP to Golang
-
Best sqlc alternative for dynamic queries?
I use xo https://github.com/xo/xo . It generates CRUD queries by default so i don't have to write basic queries and it has option to write complex queries like sqlx. Only issue is it is not well documented.
-
Why SQL is right for Infrastructure Management
SQL is an old, irregular language to work with, but it is better known than HCL and SQL already has it's own Pulumi/CDK in the form of every ORM with introspection (like Javascript's Prisma, Python's Django, Go's XO etc) and QueryBuilder (LINQ, Knex, etc) in whatever programming language you prefer. You probably already know it.
-
Help with XO's new templating, how to access fields and params
I am using the library xo to generate basic data retrieval code from a sql schema for a crud api.
-
Matt Mueller: Building Modern Web Applications Faster With Bud
Sorry for the confusion, we're not generating the database client itself, more like generating an ORM around a database client. The ORM takes these database clients as dependencies. It's very similar to the way XO works with it's multi-database support: https://github.com/xo/xo/tree/master/_examples/northwind
-
What’s your preferred setup to work with SQL DB (without ORM) ?
i use xo . Reason is i prefer designing schema first and creating golang scaffolding later. Xo takes schema and gives me basic create/update/delete operations by default and i can also generate gocode for any sql queries that i write.
- Show HN: A Go framework for your projects
-
sqlc: Generating go code from sql statements
Thanks for sharing your thoughts! I see that it works best with Postgresql. The other commenter mentioned https://github.com/xo/xo for MySql which might work well.
-
Show HN: A Full-Stack Web Framework Written in Go
Thanks for your comment and question @onionisfruit. Top-notch handle too!
>> What are your plans for models and persistence?
I haven't worked out all the details, but it's going to be some blend of https://github.com/xo/xo and https://sqlc.dev/.
Design goals:
1. High-level, type-safe "ORM" that's generated from your database schema.
What are some alternatives?
migra - Like diff but for PostgreSQL schemas
sqlc - Generate type-safe code from SQL
bytebase - The GitLab/GitHub for database DevOps. World's most advanced database DevOps and CI/CD for Developer, DBA and Platform Engineering teams.
igor - igor is an abstraction layer for PostgreSQL with a gorm like syntax.
OpenDBDiff - A database comparison tool for Microsoft SQL Server 2005+ that reports schema differences and creates a synchronization script.
SQLBoiler - Generate a Go ORM tailored to your database schema.
skeema - Declarative pure-SQL schema management for MySQL and MariaDB
prometheus - The Prometheus monitoring system and time series database.
SQLMonitor - SQL Server monitor, manages sql server performance, monitor sql server processes and jobs, analyze performance, analyse system, object version control, view executing sql query, kill process / job, object explorer, database shrink/log truncate/backup/detach/attach.
BTrDB - Berkeley Tree Database (BTrDB) server
prisma-engines - 🚂 Engine components of Prisma ORM
dat - Go Postgres Data Access Toolkit