grenad
Tools to sort, merge, write, and read immutable key-value pairs :tomato: (by meilisearch)
simdutf8
SIMD-accelerated UTF-8 validation for Rust. (by rusticstuff)
Our great sponsors
| grenad | simdutf8 | |
|---|---|---|
| 6 | 15 | |
| 19 | 508 | |
| - | 1.2% | |
| 5.3 | 3.9 | |
| 14 days ago | 3 months ago | |
| Rust | Rust | |
| MIT License | GNU General Public License v3.0 or later |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
grenad
Posts with mentions or reviews of grenad.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-05-28.
-
kv-par-merge-sort: A library for sorting POD (key, value) data sets that don't fit in memory
I tried new inserting algorithms on the sorter to reduce the amount of time spent sorting but didn’t find anything better than that, would you have any idea? I gave up trying better inserting algorithms by declaring that doing one final in-memory sort was more effective that doing a lot of comparison to try inserting keys in the right order…
-
What's everyone working on this week (42/2021)?
I have finalized the latest release of grenad which introduce a lookup struct called a Cursor. Grenad is a library that provides tools to sort, merge, write, and read immutable key-value pairs.
- Announcing the v0.4 of grenad: A library that provides tools to sort, merge, write, and read immutable key-value pairs
-
What's everyone working on this week (32/2021)?
I worked on my grenad library: a library used to sort, merge, write, and read key-value pairs, used in MeiliSearch for the new indexing system. I added a lot of documentation to the crate and done a lot of improvement by reducing the amount of copied memory when merging key-value pairs.
-
What's everyone working on this week (19/2021)?
I was wondering if you could maybe point me to any blog post regarding the segment merging? I am using my grenad library, which is a simplification of my MTBL Rust port. This is a key-value store with some useful sorter helping functions, using a merge function, some compression parameters... Works great but could maybe be improved.
-
Cross-process locks between transactions in Sanakirja (Rust database backend)
I also wanted to develop an immutable key-value store with some interesting properties like compression and multi-database (like for LMDB), it is much easier to develop than LMDB, I just need to base this on something like my grenad lib plus a wrapper to support multi-databases.
simdutf8
Posts with mentions or reviews of simdutf8.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2021-08-05.
-
simdutf: Unicode validation and transcoding at billions of characters per second
That's not enough to make it interesting. There's already a porting of it in rust (https://github.com/rusticstuff/simdutf8), and inclusion in the stdlib has already been discussed: the problem is that you can't use simd in all supported targets and conditional compilation/detection is also very tricky.
-
Making Rust Float Parsing Fast: libcore Edition
No, libcore uses simple branching code at the moment, see https://github.com/rust-lang/rust/issues/68455. The issue is still actively being worked on. Note, it's not a simple drop in, and there seem to be even faster algorithms. For now there is https://github.com/rusticstuff/simdutf8.
-
What's everyone working on this week (19/2021)?
I will work on simdutf8, either
- simdutf8 v0.1.2 - Apple Silicon can get very fast UTF-8 validation too
-
simdutf v0.1.1 - A small step for semver, one giant leap for performance.
Now I have to benchmark again, it might be negligable. See also the discussion for this pull request.
-
Show HN: High-speed UTF-8 validation in Rust
Check the benchmarks section (https://github.com/rusticstuff/simdutf8#Benchmarks), second table. simdutf8 is up to 28 % faster on my Comet Lake CPU. However with pure ASCII clang does something magical with simdjson and it beats my implementation by a lot. GCC-compiled simdjson is slower all around except for a few outliers with short byte sequences.
The algorithm is the one from simdjson, the main difference is that it uses an extra step in the beginning to align reads to the SIMD block size.
- High-speed UTF-8 validation in Rust
What are some alternatives?
When comparing grenad and simdutf8 you can also consider the following projects:
OpenVehicleDiag - A rust based cross-platform ECU diagnostics and car hacking application, utilizing the passthru protocol
sqloxide - Python bindings for sqlparser-rs
chartered - ✈️ a private, authenticated, permissioned cargo registry
simdutf - Unicode routines (UTF8, UTF16, UTF32) and Base64: billions of characters per second using SSE2, AVX2, NEON, AVX-512, RISC-V Vector Extension. Part of Node.js and Bun.
rescrobbled - MPRIS music scrobbler daemon
cxx - Safe interop between Rust and C++
link-to-notion - Quick add a link to a page within Notion app
encoding_rs - A Gecko-oriented implementation of the Encoding Standard in Rust
feel
bumpalo - A fast bump allocation arena for Rust
fast_float - Fast and exact implementation of the C++ from_chars functions for number types: 4x to 10x faster than strtod, part of GCC 12 and WebKit/Safari