DreamFactory VS postgresql-replicant

DreamFactory comes to mind - https://www.dreamfactory.com/ (https://github.com/dreamfactorysoftware/dreamfactory) which allows you to create apis from databases.

I feel very uneasy about this, and I am not sure why. I feel like the "meat and potatoes" of any application is its data and backend code, not the frontend. Frontends can change among clients, or even be given up altogether (just use a CLI/API interface for example) but the backend business logic is what defines a program. An app is what it _does_, not what it _looks like_.

This approach has been tried several times over the past few years: I first used DreamFactory[1] several years ago for a small production app (less than a few thousand users) and now I see Supabase [2] trying the same. But in my experience these don't go past very limited internal enterprise use-cases, MVPs and toy frontend apps.

[1] https://www.dreamfactory.com

You can use a library like https://github.com/agentultra/postgresql-replicant or similar to hook in a data plane and use PostgREST as a data plane.

Your business logic works on the event stream that comes from the WAL.

I wrote postgresql-replicant [0] in order to do this. It’s not a novel idea. It lets you read the logical replication stream.

On top of this library you can do all sorts of things: replicate data to Kafka or SQS, write a web socket server to publish changes to clients, etc.

[0] https://github.com/agentultra/postgresql-replicant

https://github.com/agentultra/postgresql-replicant — stream delta changes from a PostgreSQL database. Not a novel concept or implementation but this library doesn’t exist in Haskell and that’s a shame because Haskell has lots of great libraries for working with streams.

https://github.com/agent ultra/DataVersion — type-directed data migration library. I needed to migrate some data in a document database that had no schema with years of data in it to a format that had a schema. I leveraged some other stuff to achieve it but this library was a key piece to being confident that the transformations were correct and complete. Migrated the database and backfill it with a Lambda function using this library.

Those are just the most recent ones.

On my stream at https://twitch.tv/agentultra I've been working on a library to stream changes out of a PostgreSQL database. We've had to figure out how to use low-level libpq bindings, concurrency primitives, serializing and deserializing binary messages, etc. You can find the library here: https://github.com/agentultra/postgresql-replicant

You can star the repo and report issues at: https://github.com/agentultra/postgresql-replicant -- new contributors are welcome! I'm still working out a contribution guide, it's still super early on the project. I can be found here and on the FP Slack (https://fpchat-invite.herokuapp.com/) and on the FP Zulip (https://funprog.zulipchat.com)