cargo-semver-checks VS octosql-plugin-random_data

If you'd like to dig deeper, here are some links:

- cargo-semver-checks: https://github.com/obi1kenobi/cargo-semver-checks

- Trustfall query engine, which powers cargo-semver-checks: https://github.com/obi1kenobi/trustfall

- Trustfall playground, where you can query Rust library APIs in your browser -- for example, "which structs in `itertools` are importable by more than one path": https://play.predr.ag/rustdoc#?f=2&q=*3-Structs-importable-f...

- 10min conference talk on Trustfall: https://www.hytradboi.com/2022/how-to-query-almost-everythin...

I'm also giving a talk at P99 CONF in a few months about how Trustfall's new optimizations API made cargo-semver-checks over 2300x faster: https://twitter.com/PredragGruevski/status/16893002495908003...

I strongly recommend that anybody creating new error types for public APIs read Study of std::io::Error by u/matklad to see some ways that error types can [need to] be future-proofed. I don't know if cargo-semver-checks can catch these issues when they're generated by a macro, but it'd be something people using this crate should carefully look into.

cargo-semver-checks will implement a check for cases like this, and many other hazards like it. The check will be major, or minor, or just a hazard — whatever the overall community decides is right.

Full release notes: https://github.com/obi1kenobi/cargo-semver-checks/releases/tag/v0.18.0

Release notes, TL;DR: Rust 1.65+ only, no more false positives due to moved+re-exported items.

Hey!

> I think the main fundamental difference is that this wants all of the data upfront in a data file.

Absolutely not! Moreover, OctoSQL can push down predicates to databases so that it only has to download a small subset of the table, if the datasource and query allow it.

> Very easy to model HTTP APIs as a table.

"Very easy" is relative, but you can take a look at the random_data[0] datasource which is exactly this. I'm also planning to add a GitHub datasource fairly soon. That said, there is Steampipe[1] for which this is the main use case afaik (hitting API's and exposing them as tables through Postgres FWD's written in Go), so it might be a smoother and more polished experience. There's also tons of plugins already available for it.

> Easy to model basically anything as a table for example files on my filesystem.

Yep, definitely. That's the idea behind OctoSQL. Strive to create a tool for easily exposing anything through SQL (like your machine's processes list, an API, and join that with a file, or database). There's still lot's of documentation work left to do though, in order to make the plugin authoring experience easier.

> A decent query planner so that I can avoid expensive things (like API calls) if I can determine if I need the object based on something cheaper (like a local disk access).

Probably depends on the use-case, and it sometimes needs you to be fairly explicit, but OctoSQL does in fact do that. It will push down predicates to underlying databases, which means joining something small with something very big (while only taking very small amounts of the latter) can be very fast with LOOKUP JOIN's.

> I want something that is easy to extend to sources that are possibly non-listable or at the very least I don't want to have all of the data available.

Doable. An example of this is the `plugins.available_versions` table[2]. It requires you to provide the plugin name as a predicate, as the versions need to be downloaded from the plugin's own repository (and listing all plugin repositories on each query isn't really what you want to be doing). You can also LOOKUP JOIN with the `plugins.available_plugins` table if that is indeed what you want.

[0]: https://github.com/cube2222/octosql-plugin-random_data

[1]: https://steampipe.io

[2]: https://github.com/cube2222/octosql/blob/main/datasources/pl...