rust-postgres VS tikv

For C there should be good options.

For Rust it doesn't appear that well-supported.

A very simple approach is to poll for changes using `pg_logical_slot_get_changes()` - that should work with any driver. That's what I used for my initial experimentation, before switching over to the streaming replication protocol for better performance.

The streaming replication protocol is not that complicated, but currently you'll have to handle some of the low-level protocol yourself, or work with some very experimental implementations. There's a project to help get you started at [1], and some more discussion at [2].

For the logical decoder, wal2json is quite nice to experiment with, but I've found pgoutput is not that complicated and gives you something closer to the raw data.

[1]: https://github.com/seddonm1/logicaldecoding/

[2]: https://github.com/sfackler/rust-postgres/issues/116

This is a bit of a rant so please bear with me. I wrote a small utility program a long time ago that used this version of the postgres crate

En este post aprendermos a usar Rust, Tonic y la crate gRPC, y implementaremos un CRUD con Postgresql database.

I can understand that this can be frustrating and I know that the situation there is not ideal for diesel. There are certainly things to improve there by either providing a bundling support which builds the native library as part of the normal build process or by implementing a pure rust connection implementation. Both is possible with diesel, but requires some work. At least the pure rust connection implementation is something that can be provided by a third party crate now with upcoming diesel 2.0 release. If you are interested in that checkout this and this issue. As for the bundling support: This requires changes in the mysqlclient-sys and pq-sys crates. Again help there is welcome. In the end it makes me sad that some people have repeating decided that a solution to this problem is to write just another crate instead of helping to fix these issues. This just results in everyone have more work to do, as there are now two non-perfect solutions instead of having one slightly improved solution.

We created this little library to have a fully native type for Postgres Numerics with the rust-postgres (https://github.com/sfackler/rust-postgres) library.

The problem with a crate like postgres is that you have to define the types of the query at compile-time. And if you use the simple query protocol in postgres, you just get a bunch of strings, i.e. no proper typing at all. However, for maximal flexibility arbitrary queries should work in pigeon, without knowing the database schema.

Some nifty things about usual: - It's a generic wrapper over any SQL "row" object. The first implementation that's provided is for tokio-postgres, but traits are available to implement over whatever you'd like. - It provides static typing for partial queries. That is, it supports fetching a subset of fields from a row and makes a unique type for the return value. This gives you some neat-o type safety like this:

In this post, we will learn how to use Rust Tonic gRPC crate. We will learn how to implement CRUD with Postgresql database.

(see: https://github.com/sfackler/rust-postgres/blob/e15c9b1415f69821799f1370246581c1600a6196/postgres-protocol/src/types/mod.rs#L137)

Early monitoring revealed that when clients grew to 2,000–3,000, TiKV nodes quickly hit performance limits. We found that:

https://github.com/tikv/tikv - Distributed, transactional, key value store. Originally by the same company as TiDB.

In theory you could use it as an in-process KV store similar to how SQLite provides an in process sql database, but the api is far from ergonomic for that use case.

If you’re looking for the TiKV project link: https://github.com/tikv/tikv

Restate is built as a sharded replicated state machine similar to how TiKV (https://tikv.org/), Kudu (https://kudu.apache.org/kudu.pdf) or CockroachDB (https://github.com/cockroachdb/cockroach) are designed. Instead of relying on a specific consensus implementation, we have decided to encapsulate this part into a virtual log (inspired by Delos https://www.usenix.org/system/files/osdi20-balakrishnan.pdf) since it makes it possible to tune the system more easily for different deployment scenarios (on-prem, cloud, cost-effective blob storage). Moreover, it allows for some other cool things like seamlessly moving from one log implementation to another. Apart from that the whole system design has been influenced by ideas from stream processing systems such as Apache Flink (https://flink.apache.org/), log storage systems such as LogDevice (https://logdevice.io/) and others.

We plan to publish a more detailed follow-up blog post where we explain why we developed a new stateful system, how we implemented it, and what the benefits are. Stay tuned!

I know that very similar functionality to this is in TiDB Serverless ( https://tidbcloud.com ). TiDB is a distributed relational database. It uses TiKV ( which is a key/value engine ) as the storage engine. You could use SQL to access your K/V records. There is ongoing work in TiKV to support S3 directly as the storage backend ( https://github.com/tikv/tikv/issues/6506 ) .

TiKV is basically a layer on top of rocksdb https://github.com/tikv/tikv/blob/956610725039835557e7516828...