sqlite_fdw VS honeysql

https://github.com/pgspider/sqlite_fdw

Postgres has foreign data wrappers which can kinda achieve this(see also: https://github.com/pgspider/sqlite_fdw ), but ive been surprised by how well sqlite is already supported by many tools : sometimes it comes for free as its used in testing - hooking up stuff like litestream can work really wel l too

Another option might be the SQLite FDW

I've not done this but it's intriguing; potentially a best-of-all-worlds solution.

I think "proper" automatic replication is not possible given the mismatch between Postgres and SQLite - not everything in Postgres maps to a thing that is possible in Postgres.

That said, there are a variety of ways to get data out of Postgres, and a variety of ways to get things into SQLite.

You could periodically export CSVs or whatever from Postgres and periodically import them into SQLite.

Or you could do a more realtime sync by using a Postgres foreign data wrapper like this one: https://github.com/pgspider/sqlite_fdw which would let you write directly to SQLite. Combine that with database triggers on the Postgres side of things and you've got something pretty close to realtime replication.

Those sorts of solutions wouldn't be as robust as "real" replication, but could be very useful for a lot of scenarios. You could have Postgres importing gobs of data, and "replicating" it over to your reporting server which uses SQLite as a data source.

pgadmin4 does not support it, but you can try sqlite_fdw

If you encounter limitations posed by SQLite it will be easy to migrate to PostgreSQL. You can even open the SQLite database in PostgreSQL.

I recall using korma way back I and I don’t recall it being terrible but I would say https://github.com/seancorfield/honeysql has very much superseded it by this point… (but I can see how that might not be obviously clear if one is to look at superficial metrics like GitHub stars for example…)

Joins can certainly work in a data format like YAML. For an example, see Honey SQL from the Clojure community [0] (though without something to contrast strings like Clojure's keywords, you miss out on the automatic parameterization).

You mentioned moving JOINs around, so I'll mention that if represented as structured data, you can move any of the top level components around, so you could more closely follow the "true order of SQL" [1]. For example, I would love to be able to put FROM before SELECT in all or almost all cases. There's also being able to share and add to something like a complicated WHERE clause, where essentially all programming languages have built-in facilities for robustly manipulating ordered and associative data compared to string manipulation, which is not well-suited for the task.

Now don't get me wrong, I don't particularly care for YAML (though it doesn't bother me that much), but as someone who's done their fair share of programmatic SQL creation and manipulation in strings, not having a native way to represent SQL as data is a mistake in my opinion.

0: https://github.com/seancorfield/honeysql#big-complicated-exa...

In Clojure-land, we are also using HoneySQL [1] which has similar characteristics. You are still working within SQL semantics so it's a bit more complicated, but we are doing great complicated things with just maps, no API necessary.

[1] https://github.com/seancorfield/honeysql

using honeysql we can make structured queries as well

As someone else already pointed out, next.jdbc is good for database connectivity (for Postgres and beyond). For composing the queries themselves, I strongly recommend Honey SQL. It lets you represent queries themselves as normal Clojure data structures, just vectors and maps.

Perhaps you're looking for a way of arranging SQL as an AST represented by data structures (or objects) that can be fed to a compiler. HoneySQL[0] is one such implementation of this idea and it makes your general transformation trivial for Clojure programs. You don't need to mess around with string concatenation because you have a predictable and extensible compiler for data structures (which are themselves easily composable/transformable/storable with Clojure) that you can trust to do the right thing. If you're using some weird database or need an esoteric syntax, extending the compiler to your clause is easy to do[1].

[0] https://github.com/seancorfield/honeysql

[1] https://github.com/seancorfield/honeysql#extensibility

https://github.com/seancorfield/honeysql (write SQL without having to write SQL)

I've used it from Clojure, via HoneySQL, so no ORM, no danger of SQL injection. It was really wonderful!

https://github.com/seancorfield/honeysql

I used it to quickly iterate on the development of migration SQL scripts for a MySQL DB, which was running in production on RDS.

I might have switched to H2 DB later, because that was more compatible with MariaDB, but I could use the same Clojure code, representing the SQL queries, because HoneySQL can emit different syntaxes.