transit-lang-cmp VS hashbrown

> The axum::debug_handler macro is invaluable to debug type errors (there's some with axum too), like for example, accidentally having a non-Send type slip in.

Heh, yeah. For my recent project where I explored implementing the same little app in a few different languages[0], I chose Axum for the rust version.

The whole "extractor" system was pretty magical, and when I had this exact issue (non-Send argument), the compiler error was totally useless. I did see the docs about adding this extra macro crate for error messages but it seemed like a bit of a red flag that the framework was going against the grain of the language. Still, on the whole, I did enjoy working with Axum.

[0] https://github.com/losvedir/transit-lang-cmp

This is great! Just pushed up a commit that uses it and updated the benchmarks[0]. I'm seeing a 1.6X - 2X improvement in overall performance. Not bad for a drop-in replacement. And since it's based on serde, I trust it, and I feel like trying out a different JSON library is within scope for me of not just "gaming the benchmarks", as this is actually something I'd now consider using at work.

It's not quite as high as I was seeing with `jiffy` (3,800 req/sec here vs 4,000+ with jiffy), but I'm not confident that was a totally fair comparison. `jiffy` doesn't integrate as nicely with Phoenix, so I was just calling `:jiffy.encode(...)` in the controller and then doing a `text(...)` response. I need to double-check if `json(...)` is doing more work here.

[0] https://github.com/losvedir/transit-lang-cmp/commit/140d693b...

Nowadays, I reached out for some benchmark results. Scala is slower than Java and Kotlin. Can you explain it? https://github.com/losvedir/transit-lang-cmp https://github.com/kostya/benchmarks

I had a chance to update the Go code (commit) to pre-allocate the arrays based on the known length before all the appends, and saw ~30% increase in performance, with top requests per second going from about 8,600 to 11,000.

I recently did a little project to compare several languages (https://github.com/losvedir/transit-lang-cmp) so I contributed to a bunch of those points!

That's because you're looking at a wrapper around the actual implementation (which lives in an external package). Notice "use hashbrown::hash_map as base;" at the top.

There's far more unsafe there: https://github.com/rust-lang/hashbrown/blob/f2e62124cd947b5e...

You might want to check out how popular ecosystem crates do some of these things. Particularly relevant to you are probably crates providing collections, such as smallvec, hashbrown, or indexmap.

https://github.com/topics/swisstable

rust-lang/hashbrown: https://github.com/rust-lang/hashbrown

CuPy has array but not yet hashmaps, or (GPU) SIMD FWICS?

NumPy does SIMD:

Thanks, great point indeed. I am looking into this https://github.com/rust-lang/hashbrown

The way I think about it -- rather naively, I suppose -- is that I care more about the references cells make to each other than the actual grid of cells displayed on a table. The latter feels more like a "view" of the data than an actual data structure?

This also seems to align with the relative priority of (sorted from highest to lowest): figuring out the order of evaluation, calculating those evaluations, and finally displaying the results of the evaluation

updated std's hashbrown dependency to 0.14 which contains some optimizations

The std hashmap is actually very fast and uses state of the art hashmap design, namely because it's implemented by hashbrown

I believe Rust uses hashbrown as the underlying implementation now. This just calculates the number of buckets based on the number of items requested:

https://github.com/rust-lang/hashbrown/blob/009969a860290849...

Is it really the case that rehashing can guarantee that the number of buckets allocated will be sufficient for any given set of keys? In principle you could fail to rehash in a way that reduces collisions after k attempted rehashings.

For example, since the hashbrown crate is marked with #![no_std], it can be used as a dependency for the standard library.

To implement something that cannot be expressed in safe Rust, or at least cannot be expressed succinctly in safe Rust, like fundamental datastructures. The hashbrown crate contains a lot of unsafe code, but it's such high quality that it's now the backing implementation for std::collections::HashMap.