pgroll VS pgrx

My thoughts exactly. It's surprising that external online schema change tools for Postgres have only become a thing fairly recently! The only two I'm aware of are:

* pgroll: Written in Golang, first commits June 2023. https://github.com/xataio/pgroll

* pg-osc: Written in Ruby, first commits Dec 2021. https://github.com/shayonj/pg-osc

Meanwhile over in the MySQL and MariaDB ecosystem, external OSC tools have been around for quite some time, starting with oak-online-alter-table over 15 years ago. The most popular options today are pt-online-schema-change or gh-ost, but other battle-tested solutions include fb-osc, LHM, and the latest entry Spirit.

I thought I recognized xataio - they submitted pgroll a few months back https://news.ycombinator.com/item?id=37752366 (https://github.com/xataio/pgroll - Apache 2)

At the start of October we released pgroll, an open source tool for zero-downtime, reversible schema migrations for Postgres.

If you have any suggestions or questions, please open an issue in our GitHub repo, reach out to us on Discord or follow us on X / Twitter. We'd love to hear from you and keep you up to date with the latest progress on pgroll.

Any pgroll operations[0] that require a change to an existing column, such as adding a constraint, will create a new copy of the column and backfill it using 'up' SQL defined in the migration and apply the change to that new column.

There are no operations that will modify the data of an existing column in-place, as this would violate the invariant that the old schema must remain usable alongside the new one.

[0] - https://github.com/xataio/pgroll/tree/main/docs#operations-r...

This is a fantastic article! It shows that even simple migrations (like adding or removing a column) can be quite tricky to deploy in concert with the application deployement.

We (at Xata) have tried for a while to come up with a generic schema migration system for PostgreSQL that makes this easier. We ended up using views and temporary columns in such a way that we can provide both the "old" and the "new" schema simultaneously. Up/down triggers convert newly inserted data from old to new and the other way around. This also has the advantage the it can do rollbacks instantly by just dropping the "new" view.

We were just planning to announce this as an open source project this week, but actually it is already public, so if you are curious: https://github.com/xataio/pgroll

Consider also the companies and work behind pgrx [0] and pgzx [1]:

[0] https://github.com/pgcentralfoundation/pgrx

[1] https://github.com/xataio/pgzx

If you like this (I do very much), you might also like pg_idkit[0] which is a little extension with a bunch of other kinds of IDs that you can generate inside PG, thanks to the seriously awesome pgrx[1] and Rust.

[0]: https://github.com/VADOSWARE/pg_idkit

[1]: https://github.com/pgcentralfoundation/pgrx

(disclosure, i work at supabase and have been developing TLEs with the RDS team)

Trusted Language Extensions refer to an extension written in any trusted language. In this case Rust, but it also includes: plpgsql, plv8, etc. See [0]

> PL/Rust is a more performant and more feature-rich alternative to PL/pgSQL

This is only partially true. plpgsql has bindings to low-level Postgres APIs, so in some cases it is just as fast (or faster) than Rust.

> Building a vector index (or any index for that matter) inside Postgres is a more involved process and can not be done via the UDF interface, be it Rust, C or PL/pgSQL

Most PG Rust extensions are written with the excellent pgrx framework [1]. While it doesn't have index bindings right now, I can certainly imagine a future where this is possible[2].

All that said - I think there are a lot of hoops to jump through right now and I doubt it's worth it for the Latern team. I think they are right to focus on developing a separate C extension

[0] TLE: https://supabase.com/blog/pg-tle

[1] pgrx: https://github.com/pgcentralfoundation/pgrx

[2] https://github.com/pgcentralfoundation/pgrx/issues/190#issue...

I’m currently playing with PostgreSQL, foreign data wrappers, and pgrx rust extensions. My development experience has been surprisingly smooth and enjoyable.

My main issue is that joins will be processed locally, so all the foreign data will be fetched before the join happens. But otherwise basic CRUD is easy.

https://wiki.postgresql.org/wiki/Foreign_data_wrappers

https://github.com/pgcentralfoundation/pgrx

https://github.com/supabase/wrappers

I think maybe what you’re really looking for are the files here: https://github.com/pgcentralfoundation/pgrx/tree/c2eac033856...

Those are the internals we currently expose as unsafe “sys” bindings.

As we/contributors identify more that are desired we add them.

pgrx’ focus is on providing safe wrappers and general interfaces to the Postgres internals, which is the bulk of our work and is what will take many years.

As unsafe bindings go, we could just expose everything, and likely eventually will. There’s just some practical management concerns around doing that without a better namespace organization —- something we’ve been working.

The Postgres sources are not small. They are very complex, inconsistent in places, and often follow patterns that are specific to Postgres and not easy to generalize.

If you’ve never built an extension with pgrx, give it a shot one afternoon. It’s very exciting to see your own code running in your database.

pgrx is one of the greatest enabling innovations in the PG ecosystem in a long time.

Awesome to see so many high quality extensions come out of it.

https://github.com/pgcentralfoundation/pgrx

If you're interested in what my Threadripper 3970X does with it, there's some numbers in this PR: https://github.com/tcdi/pgrx/pull/1147