capnproto-rust VS crates.io

The only hurdle I have is that while the documentation is extensive it is a little confusing in places and mainly focuses on C++ and the C++ RPC system which is a little different to the Rust code. There are Rust examples in the official repo which I will try and leverage here.

capnproto-rust is the official Rust implementation.

Hi, you seem to know a bit about Sanakirja!

It stores 4kb blobs, right? Does Pijul first parses the data (copying it to other allocations), or uses the data as is? I mean, there are some libraries like cap'n'proto[0] and rkyv[1] that can directly use the file contents as an in-memory data structure, I was wondering if Pijul did anything like that.

I mean, is this btree page [2] stored exactly like this on disk, and does Pijul exploits that to avoid further copying data?

(I guess there's a trouble with compression there: to decompress you really need to write in another buffer)

Also, is the I/O done with something that prevent userspace copies like mmap or io_uring, or does it eventually calls read() to copy the data to its own buffer?

I want to build something like Sanakirja, but with those features, so I'm wondering if there's any overlap.

[0] https://github.com/capnproto/capnproto-rust

[1] https://github.com/rkyv/rkyv

[2] https://docs.rs/sanakirja-core/latest/sanakirja_core/btree/p...

Rust has a rich ecosystem of frameworks and libraries that let you read, parse, and manipulate text files, interact with cloud services and databases, and perform any other job that your project's development workflow may require. And because of its strong typing and tight memory management, you are much less likely to write programs that behave unexpectedly in production.

All the code discussed in this article can be accessed through this repository. For integration with existing projects consider using keyword_extraction crate available on crates.io.

So, be sure to double-check your critical libraries and be sure their alternatives exist in the Rust ecosystem. There’s a good chance the crates you need are available in Rust's crates.io repository.

The previous section was very simple, this section is also very simple but introduces us to cargo which is Rust's package manager, as a JS dev my mind goes straight to NPM.

Now, there has to be a place where all these packages come from. Similar to npmjs registry, where all node packages are registered, stored and retrieved, Rust also has something called crates.io where many helpful packages and dependencies are registered.

Before proceeding, let's check https://crates.io/, the official Rust package registry.

The most thrilling aspect has been the joy of writing the backend. It's like every struct, enum, and method in Rust forms this interconnected Multiverse of code , which you can see in crates.io which is best Documentation experience I Ever Had.

5. Crates.io

When using crates from eg. crates.io, and also things like std and core

Vendoring your packages was very tedious to even remotely get to work with Cargo. I spent a very long time getting Cargo to work together with cargo-local-registry. We vendor crates from crates.io and a custom internal registry.