Our great sponsors
-
edgedb
A graph-relational database with declarative schema, built-in migration system, and a next-generation query language
-
Prisma
Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Q: What are your plans for sharding / scale-out?
At Notion, we run ~480 logical schemas spread across ~32 Postgres databases [1]. Our data model [2] (sorry for the blog post spam) has a recursive / graph-like structures, and we make use of columns or jsonb attributes like `{ table: Table, id: UUID }` which sounds like your "references" feature. It seems like EdgeDB's model lines up well with how we already use our database.
I did a quick cmd-f here, on the linked announcement, and in your docs looking for "shard" and "scale" but didn't find any relevant results.
Q: Do you have plans to support EdgeQL embedding or SQLite?
I am always looking for a way to compile "better than SQL" languages down to SQL. I like Datalog in this area because it's a composable way to define relationships/facts/derivations but no one is putting serious business effort to this idea (honorable mention to logica [3]). EdgeQL also fits the bill -- queries aren't logical, but they are composable -- plus looks easier to teach than most Datalog variants.
I took a peek in the repo and saw that a few EdgeQL components are written in Rust [4]; are you considering porting more logic to Rust? That would make EdgeQL much more embeddable - it could run in WASM or linked into an Android/iOS app. My pie in the sky dream is to use a single composable query/logic language to define all my relations and queries, and then compile that stuff so it works the same on both the client, server DB, and data streams (for incremental materialized views, ideally on both client & server).
If EdgeDB had a Lite version that ran on SQLite, we'd be 66% of the way there.
(To get the materialized view bits on the server, a mad scientist might already be able to point EdgeDB at Materialize [5])
[1]: https://www.notion.so/blog/sharding-postgres-at-notion
[2]: https://www.notion.so/blog/data-model-behind-notion
[3]: https://opensource.googleblog.com/2021/04/logica-organizing-...
[4]: eg https://github.com/edgedb/edgedb/tree/master/edb/edgeql-pars...
[5]: https://materialize.com/docs/overview/architecture/
do you have plans to integrate with prisma? it's almost a DIRECT translation of how the query is expressed in Prisma compared to edgedb. Googling quickly, it seems there is a ticket already. I wonder if this could be a low hanging fruit and a great growth channel since prisma is very popular.
https://github.com/prisma/prisma/issues/10210
>> We shall do better than SQL
> The EdgeQL language looks cool, and I'm sure querying via a graph structure makes certain problems easier in some use cases. However as much as people have complained about SQL, it's just so ubiquitous there needs to be a very good reason to switch away from it. Not having to write joins isn't really a good enough reason, in my opinion.
Oh, it goes much deeper than not writing joins. There's no single ORM out there that can implement a TypeScript query builder like ours, see the example in [1]. This is only possible because of EdgeQL composability, but that composability required us to rethink the entire relational foundation.
> > The true source of truth
> I'm not sure why this means EdgeDB is better. <..>
This section implies that EdgeDB's schema allows to specify a lot of meta / dynamically computed information in it. And soon your access control policies. Take a look at the work-in-progress RFC [2] [3] to see how this is more powerful, then say, Postgres' row level security.
> > Not just a database server
> It sounds like they have a solid client, which is awesome.
Also lightweight connections to the DB so that you can have thousands of concurrent ones without load balancers, built-in schema migrations engine, and many other things. In fact we have so much that it's challenging what to even highlight in a blog post like the 1.0 announcement.
> Cloud-ready database APIs
> This used to be true, but is definitely no longer true. Cloud-native databases are everywhere and incredibly common. See any major cloud, https://www.cockroachlabs.com/, or any of the tons of other database solutions.
Not to pick on CockroachDB (they have an amazing product and company, we love them), but you should benchmark local install of Postgres and Cockroach to see yourself that scalability still has a significant cost in performance.
[1] https://www.edgedb.com/blog/edgedb-1-0#not-just-a-database-s...
[2] https://github.com/edgedb/rfcs/pull/49
[3] https://github.com/edgedb/rfcs/pull/50/files
>> We shall do better than SQL
> The EdgeQL language looks cool, and I'm sure querying via a graph structure makes certain problems easier in some use cases. However as much as people have complained about SQL, it's just so ubiquitous there needs to be a very good reason to switch away from it. Not having to write joins isn't really a good enough reason, in my opinion.
Oh, it goes much deeper than not writing joins. There's no single ORM out there that can implement a TypeScript query builder like ours, see the example in [1]. This is only possible because of EdgeQL composability, but that composability required us to rethink the entire relational foundation.
> > The true source of truth
> I'm not sure why this means EdgeDB is better. <..>
This section implies that EdgeDB's schema allows to specify a lot of meta / dynamically computed information in it. And soon your access control policies. Take a look at the work-in-progress RFC [2] [3] to see how this is more powerful, then say, Postgres' row level security.
> > Not just a database server
> It sounds like they have a solid client, which is awesome.
Also lightweight connections to the DB so that you can have thousands of concurrent ones without load balancers, built-in schema migrations engine, and many other things. In fact we have so much that it's challenging what to even highlight in a blog post like the 1.0 announcement.
> Cloud-ready database APIs
> This used to be true, but is definitely no longer true. Cloud-native databases are everywhere and incredibly common. See any major cloud, https://www.cockroachlabs.com/, or any of the tons of other database solutions.
Not to pick on CockroachDB (they have an amazing product and company, we love them), but you should benchmark local install of Postgres and Cockroach to see yourself that scalability still has a significant cost in performance.
[1] https://www.edgedb.com/blog/edgedb-1-0#not-just-a-database-s...
[2] https://github.com/edgedb/rfcs/pull/49
[3] https://github.com/edgedb/rfcs/pull/50/files
I'm currently investigating whether Redis Bloom [1] could be a good tool for similar requirement.
[1] https://github.com/RedisBloom/RedisBloom
It's available as a set of Markdown files in a git repo, if that helps: https://github.com/edgedb/easy-edgedb
they also wrote uvloop [0] which is fantastic and advances the cutting edge of what can be done with modern asyncio-based Python. I saw a ~3x improvement in the throughput of a microservice I wrote when I first tried it out years ago. currently at $dayjob we just use it by default in every Python service, whether or not we expect that service to be performance-critical.
0: https://github.com/MagicStack/uvloop
This looks pretty interesting. I'd be interested to hear when the Rust bindings will be production ready: https://github.com/edgedb/edgedb-rust
They have a golang client. And there was an attempt to write a Java client[1]. Someone could help ShaileshSurya (the java client maintainer)
[1] https://github.com/ShaileshSurya/edgedb-java
Related posts
- EdgeDB – A graph-relational database with declarative schema
- Question about custom properties querying with the query builder
- Embeddable, Distributed In-Memory datastore compatible with Redis clients
- Gnet is the fastest networking framework in Go
- Best Practice: use the same datatypes for comparisons, like joins and foreign keys