grenad
lineiform
Our great sponsors
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
grenad
-
kv-par-merge-sort: A library for sorting POD (key, value) data sets that don't fit in memory
I tried new inserting algorithms on the sorter to reduce the amount of time spent sorting but didn’t find anything better than that, would you have any idea? I gave up trying better inserting algorithms by declaring that doing one final in-memory sort was more effective that doing a lot of comparison to try inserting keys in the right order…
-
What's everyone working on this week (42/2021)?
I have finalized the latest release of grenad which introduce a lookup struct called a Cursor. Grenad is a library that provides tools to sort, merge, write, and read immutable key-value pairs.
- Announcing the v0.4 of grenad: A library that provides tools to sort, merge, write, and read immutable key-value pairs
-
What's everyone working on this week (32/2021)?
I worked on my grenad library: a library used to sort, merge, write, and read key-value pairs, used in MeiliSearch for the new indexing system. I added a lot of documentation to the crate and done a lot of improvement by reducing the amount of copied memory when merging key-value pairs.
-
What's everyone working on this week (19/2021)?
I was wondering if you could maybe point me to any blog post regarding the segment merging? I am using my grenad library, which is a simplification of my MTBL Rust port. This is a key-value store with some useful sorter helping functions, using a merge function, some compression parameters... Works great but could maybe be improved.
-
Cross-process locks between transactions in Sanakirja (Rust database backend)
I also wanted to develop an immutable key-value store with some interesting properties like compression and multi-database (like for LMDB), it is much easier to develop than LMDB, I just need to base this on something like my grenad lib plus a wrapper to support multi-databases.
lineiform
-
JITting functions in Rust for runtime performance flexibility
Sounds similar to lineiform. Which isn't all that stable or actively developed, but it is a cute approach to writing a meta-jit in rust. It's a weird approach, but IMO it's worth more experimentation.
-
What’s everyone working on this week (9/2022)?
Working on Lineiform, my meta-JIT library, some more.
-
Lineiform, a meta-JIT library for Rust interpreters
In response to Cranelift, switching to my own Tangle IR won't be using Cranelift at all (it uses raw dynasm-rs for emitting instructions). I go into a bit in https://github.com/chc4/lineiform/issues/19, but Cranelift specifically has some rules about iflags, the type they use to conceptualize processor flags effects (e.g. add's carryout or overflow). You can only have one iflags value live at a time, and it can't overlap with any other math operation. This is a problem because the x86 we're lifting doesn't always follow that rule, so if we just emit Cranelift as we go it will panic and say we built an invalid function.
I go into a bit in https://github.com/chc4/lineiform/issues/19, but it's less a problem with its optimizer and more a problem with its IR constraints. Cranelift specifically has some rules about `iflags`, the type they use to conceptualize processor flags effects (e.g. add's carryout or overflow). You can only have one `iflags` value live at a time, and it can't overlap with any other math operation. This is a problem because the x86 we're lifting doesn't always follow that rule, so if we just emit Cranelift as we go it will panic and say we built an invalid function.
The iflags design in general is kinda awkward too, and was being rethought a few months ago when I was first getting this working; I think they're planning on redesigning the add carryout interface and things to be slightly more streamlined. I suspect that any redesigned interface will have similar problems with mismatch between what I want from Cranelift and what 90% of other uses of Cranelfit want, though, and so I decided to just make my own IR instead.
-
What's everyone working on this week (3/2022)?
Working on the codegen backend for Lineiform again. I sketched out a plan on how to implement register allocation in a way that hopefully doesn't have horrible behavior in the majority of cases, and implemented ~half of it last week, and hopefully I'll implement the other half and instruction scheduling this week.
-
HN: == Happy New Year HN == (What is your “plans” for the new year?)
Yup, https://github.com/chc4/lineiform. It's not usable at all yet - I was building it on top of Cranelift, which turned out to be a fairly bad idea, so I'm going to have to essentially rewrite all of it with my own codegen backend I think. I've been hacking on it on and off but it's been much slower progress due to work (and writing a codegen backend is hard...)
-
What's everyone working on this week (32/2021)?
I got struck by either a very dumb or very good idea a few days ago, and finally have a working (minimal) proof-of-concept for it: Lineiform is a meta-JIT library to nearly automatically get an optimizing method JIT from a Rust interpreter. It does dynamic recompilation on closures by lifting from x86 to Cranelift IR for runtime function inlining and constant propagation.
What are some alternatives?
OpenVehicleDiag - A rust based cross-platform ECU diagnostics and car hacking application, utilizing the passthru protocol
soundfingerprinting - Open source audio fingerprinting in .NET. An efficient algorithm for acoustic fingerprinting written purely in C#.
chartered - ✈️ a private, authenticated, permissioned cargo registry
Nova - Implementation of "Ray Tracing in One Weekend": https://raytracing.github.io/books/RayTracingInOneWeekend.html
rescrobbled - MPRIS music scrobbler daemon
cranelift-jit-demo - JIT compiler and runtime for a toy language, using Cranelift
link-to-notion - Quick add a link to a page within Notion app
augmented-audio - Rust - Augmented Audio Libraries
feel
uom - Units of measurement -- type-safe zero-cost dimensional analysis
bumpalo - A fast bump allocation arena for Rust
indicatif - A command line progress reporting library for Rust