pueue VS tikv

You can probably do a good subset it in bash, it's just a nicer interface with a lot of configurability and several convenience features.

I'm generally a big fan of showing alternatives: https://github.com/Nukesor/pueue/?tab=readme-ov-file#similar...

Would you be willing to write a proper guide on how to do all of these things in bash? It would be great to have this as guide an alternative inside the Pueue wiki and link to it. It'll help people to make a more informed decision on whether they need this tool or not.

there is a command line manager for long running tasks called Pueue. It is released into Nix, Arch, Alpine, Void, etc, but not for Debian based distros. I know that releasing into Debian is a bit more challenging, but I just wanted to ask if anybody here might be interested in packaging it. Just as a disclaimer, I am not the author of this project, just a regular user.

This one? https://github.com/Nukesor/pueue

How about this: https://github.com/Nukesor/pueue/? I have it bookmarked from a thread here from few years back and never got to test it eventually, but maybe it will serve your purposes?

Multiplexing is great for your multiple outputs, but I would highly recommend using pueue & pueued for job control. Lets you organize your background jobs into groups which can be paused, resumed, etc. Also lets you act on jobs from different terminals w/the pueue interface.

pueue -- a queue for tasks, running in background

For the command that takes a long time to complete, I always use pueue to run. This thing let you run multiple commands in order and can schedule the execution later which is really helpful to my workflow.

That said, you're basically building pueue. https://github.com/Nukesor/pueue/blob/main/ARCHITECTURE.md might give you some pointers. From reading it, there seems to be a mishmash of tokio stuff, and then everything gets serialised onto an MPSC channel (that's serviced by TaskHandler, on a single thread that's also responsible for polling for finished processes etc, every 200ms).

exchange for pueue and you can even queue them up.

This is the only one I know of: https://github.com/Nukesor/pueue

I know that very similar functionality to this is in TiDB Serverless ( https://tidbcloud.com ). TiDB is a distributed relational database. It uses TiKV ( which is a key/value engine ) as the storage engine. You could use SQL to access your K/V records. There is ongoing work in TiKV to support S3 directly as the storage backend ( https://github.com/tikv/tikv/issues/6506 ) .

TiKV is basically a layer on top of rocksdb https://github.com/tikv/tikv/blob/956610725039835557e7516828...

> Many,many smart people…

If you look inside the code you can see the stated features are a result of underlying engine (TiKV [0] also in c and rust from pingcap). Surrealdb is standing on shoulders of giants at present, they are TiKV, FoundationDB and rocksdb. The feature set they mentioned mostly coming from TiKV at present.

[0] https://tikv.org/

Hi Diggsey, great question. We are currently focussed on functionality and stability, and then will draw our attention to performance. Coming this week we have a RocksDB storage implementation. We've only just launched our initial beta version, and we know there is a lot of improvement and work to be done (some of these performance issues we know about already and are on our Github issues list).

With regards to the consistency/isolation model, SurrealDB sits on top of a number of key-value stores. By using the distributed highly-available TiKV storage backend, https://tikv.org, (and we have a FoundationDB integration in the works), the database is designed to be highly-scalable and highly-available. The same guarantees (albeit just single-node, so no high-availability or scalability) will be available with the RocksDB implementation coming this week. By sitting on top of these key-value stores, SurrealDB ensures that all transactions are ACID compliant. We don't want to go for speed (for instance by writing to /dev/null) over anything, but want SurrealDB to be a reliable and performant backend for any application. Obviously we have a way to go to catch up with PostgreSQL (launched in 1996), but we will strive to get there!

If you are looking for a production ready distributed store written in Rust. Check out TiKV(https://github.com/tikv/tikv), which was also mentioned in the acknowledge section of the project's README.

There's also a full-featured distributed RDBMS called TiDB built on top of TiKV.