edgedb
trustfall
Our great sponsors
edgedb | trustfall | |
---|---|---|
19 | 24 | |
12,152 | 2,236 | |
1.2% | - | |
9.9 | 9.3 | |
1 day ago | 11 days ago | |
Python | Rust | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
edgedb
-
Beyond SQL: A relational database for modern applications
A new DB, with a new query language that's like "SQL done right"? This immediately reminded me of EdgeDB: https://edgedb.com/
Is there anyone here who knows enough about these two products to do a compare/contrast?
See also https://edgedb.com/ which is another relational database without sql
-
EdgeDB 3.0
The whole thing consists of these main parts:
1. SQL parser: https://github.com/edgedb/edgedb/tree/master/edb/pgsql/parse...
-
DuckDB 0.8.0
>relational no-sql
Do you mean something like edgeDB?[0]
Or do you mean some non-declarative language completely? I don't see the latter making much sense. The issue with SQL for me is the "natural language" which quickly loses all intended readabilty when you have SELECT col1, col2 FROM (SELECT * FROM ... WHERE 1=0 AND ... which is what edgeDB is trying to solve.
-
EdgeDB 2.0
That `sys::TransactionIsolation` doc is outdated, EdgeDB only supports the `Serializable` transaction isolation. See related discussion [1]
At this point EdgeDB supports Postgres HA passively, i.e. it will react to a failover event in your cluster via one of the documented mechanisms. Support for "active" cluster management is a planned feature too.
Finally, EdgeDB server itself is fully stateless and you can run multiple instances of it in front of the same Postgres cluster. Admittedly, we need to document this better.
Would really appreciate it if you could open issues in the main [1] repo and we'll figure it out. We're investing a ridiculous amount of time into our docs and want them to be the best possible.
-
GraphQL Is a Trap?
You have to do your own optimiser to avoid, for instance, the N+1 query problem. (Just Google that, plenty of explanations around.) Many GraphQL frameworks have a “naive” subquery implementation that performs N individual subqueries. You either have to override this for each parent/child pairing, or bolt something on the back to delay all the “SELECT * FROM tbl_subquery WHERE id = ?” operations and convert them into one “… WHERE id IN (…)”. Sounds like a great use of your time.
In the end you might think to yourself “why am I doing this, when my SQL database already has query optimisation?”. And it’s a fair question, you are onto it. Try one of those auto-GraphQL things instead. EdgeDB (https://edgedb.com) does it as we speak, runs atop Postgres. Save yourself the enormous effort if you’re only building a GraphQL API for a single RBDMS, and not as a façade for a cluster of microservices and databases and external requests.
Or just nod to your boss and go back to what being a backend developer has always meant: laboriously building by hand completely ad hoc JSON versions of SQL RBDMS schemas, each terribly unhappy in its own way. In no way does doing it manually but presenting GraphQL deviate from this Sisyphean tradition.
I read in the article that NOT having GraphQL exactly match your DB schema is a best practice. My response is “did a backend developer write this?”
-
How we sharded our test suite for 10x faster runs on GitHub Actions
Same idea, yeah. Unfortunately, in our case we couldn't use pytest due to complicated test setup, so we used a customized unittest runner instead.
-
GraphQL is now available on Supabase
EdgeDB [1] has indeed a rich GraphQL layer, but it's a very different project.
While it also builds on top of Postgres, EdgeDB replaces the entire relational database front-end. EdgeDB features a SQL replacement language called EdgeQL (analytical capabilities of SQL married with deep-fetching in GraphQL), a higher-level data model (tables -> object types), integrated migrations engine, a custom protocol with great performance & great client APIs, and many other things. Read more here [2].
(disclaimer: I'm EdgeDB co-founder)
trustfall
-
Planner programming blows my mind
Pleasantly surprised to see Predrag show up as a reviewer, but at the same time not at all surprised:
- The [Firebase technical screen](https://startupandrew.com/posts/how-firebase-interviewed-sof...) would have been much easier with something like this, as it was Just Another Optimization Problem™. Part of me wants to try it again with Picat!
- He's doing other very interesting things with programming languages, e.g.: https://github.com/obi1kenobi/trustfall
-
Ask HN: Who wants to be hired? (December 2023)
Email: [email protected]
Hi, I'm u9g. I'm looking for an internship for either this winter (in a few weeks) or a summer internship!
Some things I've done:
I integrated a [query engine for lints](https://github.com/obi1kenobi/trustfall) (and contributed optimizations) into [OXC](https://github.com/oxc-project/oxc) (a new Rust-based Javascript Linter).
I wrote [several](https://github.com/u9g/money-lens) [toy](https://github.com/u9g/quickquestion/tree/main/extension) languages and syntax highlighting for them.
I also wrote a [js-to-scheme transpiler](https://github.com/u9g/js2scheme/blob/main/example.js).
Happy to work on things compiler or database oriented, but also happy to learn something new!
-
Semver violations are common, better tooling is the answer
If you'd like to dig deeper, here are some links:
- cargo-semver-checks: https://github.com/obi1kenobi/cargo-semver-checks
- Trustfall query engine, which powers cargo-semver-checks: https://github.com/obi1kenobi/trustfall
- Trustfall playground, where you can query Rust library APIs in your browser -- for example, "which structs in `itertools` are importable by more than one path": https://play.predr.ag/rustdoc#?f=2&q=*3-Structs-importable-f...
- 10min conference talk on Trustfall: https://www.hytradboi.com/2022/how-to-query-almost-everythin...
I'm also giving a talk at P99 CONF in a few months about how Trustfall's new optimizations API made cargo-semver-checks over 2300x faster: https://twitter.com/PredragGruevski/status/16893002495908003...
-
Cargo-semver-checks: Scan your Rust crate for semver violations
No worries!
Trustfall is even more general than semver actually, one can use it to query any data source -- even HN itself:
https://play.predr.ag/HackerNews
Here's a link to that project if you'd like to check it out:
https://github.com/obi1kenobi/trustfall
I gave a 10min conference talk on it last year titled "How to query (almost) everything"
https://www.hytradboi.com/2022/how-to-query-almost-everythin...
-
We created Taxi, and why we felt the need for Another Schema Language
I've had my eye on Taxi for a while, and it's neat! I agree that the problems it aims to solve are real and painful in real life.
In my experience, I'm not sure people care about schemas or schema languages — they are just implementation details best left under the hood. This is why in my own work, I started on the query end of the spectrum instead.
This is why I built Trustfall, a query engine able to query any data source: DBs, APIs, files like PDF, CSV, or JSON, or any combination of those.
GitHub: https://github.com/obi1kenobi/trustfall
Playground showing the query "which GitHub/Twitter users comment on HackerNews stories about OpenAI?": https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVl...
- cargo-semver-checks v0.20 and Trustfall v0.4 released — semver-check up to 2354x faster
- DuckDB: Querying JSON files as if they were tables
-
Speeding up Rust semver-checking by over 2000x
Here's an example adapter for querying RSS/Atom feeds. Here's an example adapter for querying the HackerNews APIs. Both projects also include a schema and example queries which you can run.
-
Trustfall: How to Query (Almost) Everything
As I wrote in reply to your other comment, the join algorithms are the adapter's choice. I wouldn't necessarily recommend using it as a SQL replacement, though.
There's no measurable MoM growth yet. This is the first major look the broader community has had at the system outside of a relatively small conference where I gave a talk, and demos I've given to friends and colleagues. I wasn't planning on submitting to HN for another few months -- but someone else beat me to it today :)
There are production-grade adapters for HackerNews APIs and for Rust's rustdoc JSON format, used by the Playground (https://play.predr.ag/hackernews) and the Rust semver-checking linter cargo-semver-checks, respectively. There are also a few more demo adapters in the Trustfall repo itself: https://github.com/obi1kenobi/trustfall I expect the number of adapters to grow significantly in the coming months, so please stay tuned and let me know if you have datasets you'd like to try it with!
The "largest dataset" is a bit of a trick question: how big is the dataset of "all HackerNews data available via its Firebase and Algolia APIs"? Because that's what the Playground queries.
The Rust semver linter has been used by dozens if not hundreds of crates, and the JSON payloads in question there are in the 100-400MB range. The example in this blog post runs 40 quite complex Trustfall queries (they express semver rules!) over 400MB across two JSON files in 8 seconds: https://predr.ag/blog/speeding-up-rust-semver-checking-by-ov...
You can also see more real-world use cases in this talk I gave last year: https://www.hytradboi.com/2022/how-to-query-almost-everythin...
Probably around 200-300 people or so have written queries in the language, with experience levels ranging from seasoned engineers to Excel analysts and SQL analysts with no programming experience outside of that domain. The language itself is actually a refinement from an earlier open-source project I developed and open-sourced through my previous job. That open-source project was used to query everything from a multi-TB SQL cluster to APIs to ML models, and with Trustfall I've taken the opportunity to revisit and update design decisions that the previous project got to regret in retrospect.
I bumped into him and showed it to him in person one time :)
Realistically, the docs are still very sparse -- I wasn't planning on posting this for another few months, but someone else noticed my project and beat me to it!
New data providers need a schema and an Adapter implementation.
For Rust, this is the Adapter trait that data providers need to implement: https://docs.rs/trustfall_core/latest/trustfall_core/interpr...
This is the equivalent abstract class in Python: https://github.com/obi1kenobi/trustfall/blob/pytrustfall-v0....
For Python, the package docs have some (underwhelming) additional examples of building a schema and passing an adapter to use to run a query: https://pypi.org/project/trustfall/
The process in Rust is equivalent, here's the code powering the demo in the talk: https://github.com/obi1kenobi/trustfall/blob/main/demo-hytra...
I realize this is all extremely suboptimal, but like I said, I wasn't ready Please stay tuned! I'm planning to spruce all of this up in the coming weeks!
What are some alternatives?
supabase - The open source Firebase alternative.
cockroach - CockroachDB - the open source, cloud-native distributed SQL database.
neon - Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, branching, and bottomless storage.
Prisma - Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB
supabase-graphql-example - A HackerNews-like clone built with Supabase and pg_graphql
edgedb-rust - The official Rust binding for EdgeDB
postgres - Unmodified Postgres with some useful plugins
pg_graphql - GraphQL support for PostgreSQL
supabase-graphql-examp
uvloop - Ultra fast asyncio event loop.
edgedb-java
easy-edgedb - An illustrated textbook designed to be a one-stop shop for learning EdgeDB