sqldef VS kysely-codegen

Thanks! Yeah definitely agree that building out declarative table management for Postgres would be a major effort. A few open source projects I've seen in that area include:

https://github.com/sqldef/sqldef (Go)

https://github.com/bikeshedder/tusker (Python but being ported to Rust)

https://github.com/tyrchen/renovate (Rust)

https://github.com/blainehansen/postgres_migrator (Rust)

Some of these are based on parsing SQL, and others are based on running the CREATEs in a temporary location and introspecting the result.

The schema export side can be especially tricky for Postgres, since it lacks a built-in equivalent to MySQL's SHOW CREATE TABLE. So most of these declarative pg tools shell out to pg_dump, or require the user to do so. But sqldef actually implements CREATE TABLE dumping in pure Golang if I recall correctly, which is pretty cool.

There's also the question of implementing the table diff logic from scratch, vs shelling out to another tool or using a library. For the latter path, there's a nice blog post from Supabase about how they evaluated the various options: https://supabase.com/blog/supabase-cli#choosing-the-best-dif...

If you wish to auto-generate migrations, there are declarative schema change tools available for most relational databases. I'm the creator of Skeema [1] which provides them for MySQL, but there are options for other DBs too [2][3][4].

Prisma's migration system actually partially copied Skeema's design, while giving credit in a rather odd fashion which really rubbed me the wrong way: "The workflow of working with temporary databases and introspecting it to determine differences between schemas seems to be pretty common, this is for example what skeema does." [5]

While I doubt I was the first person to ever use that technique, I absolutely didn't copy it from anywhere, and it was never "pretty common". I'm not aware of any other older schema change systems that work this way.

[1] https://www.skeema.io

[2] https://github.com/djrobstep/migra

[3] https://github.com/k0kubun/sqldef

[4] https://david.rothlis.net/declarative-schema-migration-for-s...

[5] https://github.com/prisma/prisma-engines/blob/6be410e/migrat...

By the way, I have also made a repository that uses sqldef as a migration tool to make migration as easy as possible, so please take a look at it.

sqldef-gitops-cdk sqldef is an open source idempotent schema management for MySQL, PostgreSQL, and others. This repo provides a GitOps approach using AWS CDK of how you can approach database table migration.

I work heavily in this space and can add some more details :)

Tusker actually uses migra to power its functionality: https://github.com/bikeshedder/tusker#how-does-it-work

Tusker's flow is somewhat similar to sqldef https://github.com/k0kubun/sqldef , although the internal mechanics are quite different. Migra (and therefore Tusker) executes the SQL, introspects it, and diffs the introspected in-memory representation -- in other words, using the database directly as the canonical parser. In contrast, sqldef parses the SQL itself, builds an in-memory representation based on that, and then does the diff that way.

I'm the author of Skeema https://www.skeema.io which provides a similar declarative workflow for MySQL and MariaDB schema changes. Skeema uses an execute-and-introspect approach similar to Migra/Tusker, although each object is split out into its own .sql file for easier management in version control, with a multi-level directory hierarchy if you have multiple database instances and multiple schemas.

Skeema was conceptually inspired by Facebook's internal database schema change flow, as FB has used declarative schema management submission/review/execution company-wide for over a decade now. Skeema actually predates both Migra and sqldef slightly, although it did not influence them, all were developed separately.

In turn, Prisma Migrate and Vitess/PlanetScale declarative migrations were directly inspired by Skeema's approach, paradigms, and/or even direct use of source code in Vitess's case. (Although they're finally moving to a parser-based approach instead, which I recommended they do over a year ago, as it makes more sense for their use-case -- their whole product inherently requires a thorough SQL parser anyway...)

Thanks! I know of a couple Postgres tools that work in a declarative fashion: migra [1] and sqldef [2].

Migra is Postgres-specific. Its model is similar to Skeema's, in that the desired-state CREATEs are run in a temporary location and then introspected, to build an in-memory understanding of the desired state which can be diff'ed against the current actual state. (This approach was also borrowed by Prisma Migrate [3]). In this manner, the tool doesn't need a SQL parser, instead relying on the real DBMS to guarantee the CREATE is interpreted correctly with your exact DBMS version/flavor/settings.

In contrast, sqldef supports multiple databases, including Postgres and MySQL (among others). Unlike other tools, it uses a SQL parser-based approach to build its in-memory understanding of the desired state. As a DB professional, personally this approach scares me a bit, given the amount of nonstandard stuff in each DBMS's SQL dialect. But I'm inherently biased on this topic. And I will note sqldef's author is a core Ruby committer and JIT author, and is extremely skilled at parsers.

[1] https://databaseci.com/docs/migra

[2] https://github.com/k0kubun/sqldef

[3] https://github.com/prisma/prisma-engines/blob/main/migration...

Declarative schema management tools make this much easier. The concept is your schema repo just stores CREATE statements, and the schema management tool knows how to generate DDL to transition between the current state in your DB and the desired state in your repo.

I'm the author of declarative schema management tool skeema (https://www.skeema.io, for MySQL / MariaDB). Some other options in this space are sqldef (https://github.com/k0kubun/sqldef, for MySQL or Postgres) and migra (https://github.com/djrobstep/migra, for Postgres). In MS SQL Server, SSDT DACPACs are also somewhat similar.

Hello HN! I was recently introduced to Kysely (https://github.com/kysely-org/kysely), a type-safe Typescript SQL query builder, and instantly fell in love! I got inspired and wanted to make something similar for AWS DynamoDB.

Thus, I developed Tsynamo! Instead of calling it type-safe, I decided to go with type-friendly, because the library is still in an early stage, and is not 100% type-safe.

Under the hood, Tsynamo compiles the built queries into AWS SDK v3 commands. I feel that Tsynamo simplifies the AWS SDK API quite a lot since the developer doesn't have to mess around with condition/filter expressions or attribute names/values themselves, and as a bonus gets autocompletion for building the queries!

There's also a playground to test it out in your browser: https://try.tsynamo.dev. It might not have the most up-to-date API of the library in use yet, but you can get the library's main idea from there.

Since the project is still in its early stages, it doesn't yet have 100% support for all DynamoDB features, like querying indexes. The next steps will be increasing the support coverage and perhaps adding automatic type generation as inspired by kysely-codegen (https://github.com/RobinBlomberg/kysely-codegen).

Would love to get some feedback, thanks in advance!

Querying the database (PostgreSQL) should not be ground breaking. Personally I like to have full type-safety so we can easily catch bugs during the development time without introducing any tests that are just testing the data type from our datastore to match the data type our API expects. I like to go database schema first, which means that we generate types from the database schema and work with those. Any change to the schema of the database is made with SQL migrations and after that, the typescript types are regenerated. Another approach is to use a code-first tool like TypeORM or Prisma. However in my experience such tools often produce not efficient SQL queries and are less easy to extend. In my projects I use library kysely (https://github.com/koskimas/kysely) with kysely-codegen (https://github.com/RobinBlomberg/kysely-codegen) to have a full type-safe SQL builder.

The main difference to me is that with Prisma I don't need to do any SQL to get a project up and running. I can define the tables and relations from my code or from Prisma Studio, and in one swoop I get the db structure done and the types defined.

I know my way around a database, but I'd rather not leave my code editor whenever I need to add a new column to a table.

With Kysely you have to create the DB schema, and then write the types; with every change you need, you gotta do both again.

(At least this seems true by default; as the project's readme mentions, there is a code generator[1] to generate the types from the DB schema; not quite the same but at least it's better than nothing.)

Commenters say that writing complex queries with Kysely is easier, which makes me wonder if I could use Prisma except for those, since Kysely should be able to just generate the SQL query for me to handle to Prisma...

[1]: https://github.com/RobinBlomberg/kysely-codegen

kysely kysely-codegen kysely-d1 allows access to D1 while using SQL Builder.

For what it's worth, I've been using kysely-codegen to generate the types based on the database schema and it's been working great for me.

I dislike ORMs and have been using Kysely https://github.com/koskimas/kysely and https://github.com/RobinBlomberg/kysely-codegen for a small project and so far I love it. It's a mapper/query builder that is equivalent to Dapper in C# (without the query builder) but with the power of typescript's type system to enforce proper queries. You can also bypass something with the sql to write raw SQL.