ddl-diff VS trustfall

So, after digging about in the docs, I found the Best Practices[0] page which lays out what you describe. I was disappointed to find it is just using idempotent creation statements for procedures and views and putting them all in files that are re-run on each migration step-along with each new roll forward migration-so that new versions of these logic objects replace any old ones. This is not exactly something that liquidbase provides, should be easily replicatable, and I was hoping it'd do diffs to provide similar experience on tables as well.

After some web searching, I came across a project[1] that's trying to do what I describe but it appears to be dead. I'm surprised that migration tools (that I looked through) don't already support such a basic approach that I suspect I'm missing something obvious. Some fundamental block in how SQL works or maybe it doesn't work that well in practice for, in concept, it sounds easy to script using what they call a shadow db and `pg_diff`.

[0]: https://docs.liquibase.com/concepts/bestpractices.html

[1]: https://github.com/cbowdon/ddl-diff

Pleasantly surprised to see Predrag show up as a reviewer, but at the same time not at all surprised:

- The [Firebase technical screen](https://startupandrew.com/posts/how-firebase-interviewed-sof...) would have been much easier with something like this, as it was Just Another Optimization Problem™. Part of me wants to try it again with Picat!

- He's doing other very interesting things with programming languages, e.g.: https://github.com/obi1kenobi/trustfall

Email: [email protected]

Hi, I'm u9g. I'm looking for an internship for either this winter (in a few weeks) or a summer internship!

Some things I've done:

I integrated a [query engine for lints](https://github.com/obi1kenobi/trustfall) (and contributed optimizations) into [OXC](https://github.com/oxc-project/oxc) (a new Rust-based Javascript Linter).

I wrote [several](https://github.com/u9g/money-lens) [toy](https://github.com/u9g/quickquestion/tree/main/extension) languages and syntax highlighting for them.

I also wrote a [js-to-scheme transpiler](https://github.com/u9g/js2scheme/blob/main/example.js).

Happy to work on things compiler or database oriented, but also happy to learn something new!

If you'd like to dig deeper, here are some links:

- cargo-semver-checks: https://github.com/obi1kenobi/cargo-semver-checks

- Trustfall query engine, which powers cargo-semver-checks: https://github.com/obi1kenobi/trustfall

- Trustfall playground, where you can query Rust library APIs in your browser -- for example, "which structs in `itertools` are importable by more than one path": https://play.predr.ag/rustdoc#?f=2&q=*3-Structs-importable-f...

- 10min conference talk on Trustfall: https://www.hytradboi.com/2022/how-to-query-almost-everythin...

I'm also giving a talk at P99 CONF in a few months about how Trustfall's new optimizations API made cargo-semver-checks over 2300x faster: https://twitter.com/PredragGruevski/status/16893002495908003...

No worries!

Trustfall is even more general than semver actually, one can use it to query any data source -- even HN itself:

https://play.predr.ag/HackerNews

Here's a link to that project if you'd like to check it out:

https://github.com/obi1kenobi/trustfall

I gave a 10min conference talk on it last year titled "How to query (almost) everything"

https://www.hytradboi.com/2022/how-to-query-almost-everythin...

I've had my eye on Taxi for a while, and it's neat! I agree that the problems it aims to solve are real and painful in real life.

In my experience, I'm not sure people care about schemas or schema languages — they are just implementation details best left under the hood. This is why in my own work, I started on the query end of the spectrum instead.

This is why I built Trustfall, a query engine able to query any data source: DBs, APIs, files like PDF, CSV, or JSON, or any combination of those.

GitHub: https://github.com/obi1kenobi/trustfall

Playground showing the query "which GitHub/Twitter users comment on HackerNews stories about OpenAI?": https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVl...

Here's an example adapter for querying RSS/Atom feeds. Here's an example adapter for querying the HackerNews APIs. Both projects also include a schema and example queries which you can run.