pg_hexedit VS pg_net

I am the author of a tool that generates wxHexEditor tags and annotations for Postgres relation files -- pg_hexedit:

https://github.com/petergeoghegan/pg_hexedit

I've invested quite a lot of effort in it, and it would be nice to have support for multiple hex editors. That was anticipated to some degree:

https://github.com/petergeoghegan/pg_hexedit#supporting-othe...

I understand why you favor a declarative template format for describing files with tags -- that probably scales really nicely. What I'm doing is pretty grotty, but works surprisingly well in practice. I procedurally generate a description of each file in a shell script, and then open the file in wxHexEditor. I'm generating huge XML files, which is slow, but there are simple workarounds to get acceptable performance.

wxHexEditor doesn't support declarative tags. But even if it did I might not want to use them; the on-disk format of PostgreSQL is much more complicated than most file formats, and isn't supposed to be consumed by third party tools. Writing a C program that uses the struct definitions from the server itself makes the complexity quite manageable -- the tool is basically feature complete, even though I haven't spent a huge amount of time on it.

That said, it would be great if I could adapt pg_hexedit to a hex editor that had some kind of "best of both worlds" support for tags - tags that can be generated lazily and on-demand, when a portion of the file needs to be drawn or redrawn. This would be easy to adapt to -- Postgres relation files always consist of a series of 8KiB blocks/pages. My tool can easily generate tags for any single block without knowing any special context or having any expensive-to-generate state -- I just need a block number (i.e. an 8KiB-aligned byte offset).

Third, there are some reports of pg_net failing to make requests after your database transaction volume surpasses a certain threshold.

> speaks a particular API over the network

it's a interesting idea, and one of the things that we were toying with in our pg_net extension (https://github.com/supabase/pg_net). This is a "generic" async network extension, so you can fetch/put/post. It works well for APIs.

I think the generic approach works for some things where the data is less "fixed" - for example, an OpenAI API endpoint.

But for "fixed" data (data warehouses), the wrapper usually needs some custom work for security, protocols, and "push down". I'll be interested to get HN's take on this - they might have some suggestions for us for this framework

> finding the building blocks of modern applications (database, auth, functions, presence, realtime subscriptions), making them easy to use, and then sharing the source code.

Great observation!

> I’ve learned a ton just from cruising around supabase GitHub.

Glad to hear it!

> Can you say which of these new components will be open sourced?

All of these components are open source and licensed under Apache License v2.0.

> There are some other features (e.g. function hooks) that are also closed-source at the moment.

I actually worked on the initial implementation of function hooks. We've actually already open sourced both the client (see: https://github.com/supabase/supabase/tree/88bcef911669595428...) and the pg_net extension it requires (see: https://github.com/supabase/pg_net).

> Is Supabase heading for an “open core” model?

I don't think so. We want to continue to open source our projects under either MIT (client libs) and Apache License v2.0 (server libs).

> The dream would be to have a great DX experience around using insert/update triggers to call Supabase functions to run background tasks

We have something for this: Function Hooks (soon to be renamed "Async Triggers")[0]. They are still in alpha, but the extension [1] is getting close. It was important to build something which works with PG background workers so that it's non-blocking. We'll make quick progress on this now that we've released Edge Functions.

> sending notifications or updating related rows

Tune in for tomorrow's announcement - it's related.

[0] Function Hooks / Async Triggers: https://supabase.com/blog/2021/07/30/supabase-functions-upda...

[1] https://github.com/supabase/pg_net