fast_float
simdutf8
Our great sponsors
fast_float | simdutf8 | |
---|---|---|
15 | 15 | |
1,267 | 508 | |
1.9% | 1.2% | |
8.8 | 3.9 | |
29 days ago | 2 months ago | |
C++ | Rust | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fast_float
-
Parquet: More than just “Turbo CSV”
> Google put in significant engineering effort into "Ryu", a parsing library for double-precision floating point numbers: https://github.com/ulfjack/ryu
It's not a parsing library, but a printing one, i.e., double -> string. https://github.com/fastfloat/fast_float is a parsing library, i.e., string -> double, not by Google though, but was indeed motivated by parsing JSON fast https://lemire.me/blog/2020/03/10/fast-float-parsing-in-prac...
-
What do number conversions (from string) cost?
For those that don't know, gcc 12.x updated its float parsing logic to something similar to fast_float and it's about 1/6 of the cost presented here (sub 100 in the graph presented here). Strongly suggest using that library or upgrading the compiler if you need the performance.
-
Can sanitizers find the two bugs I wrote in C++?
This makes sense for integers but betware floating point from_chars - libc++ still doesn't implement it and libstdc++ implements it by wrapping locale-dependent libc functions which involves temporarily changing the thread locale and possibly memory allocation to make the passed string 0-terminated. IMO libstdc++'s checkbox "solution" is worse than not implementing it at all - user's are better off using Lemire's API-compatible fast_float implementation [0].
-
Passing Programs To A Stack Machine
I'm a bit stuck on how to do the same thing in c++, due to containers only having a single type. The very inefficient way I'm currently doing it is by passing a program as a vector of strings, and then converting the string constants to doubles with the fast_float library.
-
Parsing can become accidentally quadratic because of sscanf
Just above this comment is a merged PR, which references fast_float library: https://github.com/fastfloat/fast_float
-
Making Rust Float Parsing Fast: libcore Edition
Daniel Lemire @lemire (creator of the algorithm, author of the C++ implementation, and provided constant feedback to help guide the PR).
-
RapidObj v0.1 - A fast, header-only, C++17 library for parsing Wavefront .obj files.
And out of 6,000 lines in the file, at least 3000 are other people's code: earcut for polygon triangulation and fast_float because .obj files typically contain a lot of floating point numbers so it's important to parse them quickly.
-
First release of dragonbox, a fast float-to-string conversion algorithm, is available
How this compares to https://github.com/fastfloat/fast_float ?
-
Why is std::from_chars<float> slow?
I tried to compare it against Daniel Lemire's excellent fast_float library. Fast float took about 180ms for the same program, and all I did was change "std" namespace prefix to "fast_float". It's a factor of 12 difference, at least my machine. I tried MSVC next, and it is a lot better, but it is still ~4 times slower than fast float. AFAIK, clang currently does not implement the feature at all.
-
Iterator invalidation of std::string_view
If you don't mind a 3rd party lib until your stdlib updates, https://github.com/fastfloat/fast_float is best-in-class.
simdutf8
-
simdutf: Unicode validation and transcoding at billions of characters per second
That's not enough to make it interesting. There's already a porting of it in rust (https://github.com/rusticstuff/simdutf8), and inclusion in the stdlib has already been discussed: the problem is that you can't use simd in all supported targets and conditional compilation/detection is also very tricky.
-
Making Rust Float Parsing Fast: libcore Edition
No, libcore uses simple branching code at the moment, see https://github.com/rust-lang/rust/issues/68455. The issue is still actively being worked on. Note, it's not a simple drop in, and there seem to be even faster algorithms. For now there is https://github.com/rusticstuff/simdutf8.
-
What's everyone working on this week (19/2021)?
I will work on simdutf8, either
- simdutf8 v0.1.2 - Apple Silicon can get very fast UTF-8 validation too
-
simdutf v0.1.1 - A small step for semver, one giant leap for performance.
Now I have to benchmark again, it might be negligable. See also the discussion for this pull request.
-
Show HN: High-speed UTF-8 validation in Rust
Check the benchmarks section (https://github.com/rusticstuff/simdutf8#Benchmarks), second table. simdutf8 is up to 28 % faster on my Comet Lake CPU. However with pure ASCII clang does something magical with simdjson and it beats my implementation by a lot. GCC-compiled simdjson is slower all around except for a few outliers with short byte sequences.
The algorithm is the one from simdjson, the main difference is that it uses an extra step in the beginning to align reads to the SIMD block size.
- High-speed UTF-8 validation in Rust
What are some alternatives?
dragonbox - Reference implementation of Dragonbox in C++
sqloxide - Python bindings for sqlparser-rs
rapidobj - A fast, header-only, C++17 library for parsing Wavefront .obj files.
simdutf - Unicode routines (UTF8, UTF16, UTF32) and Base64: billions of characters per second using SSE2, AVX2, NEON, AVX-512, RISC-V Vector Extension. Part of Node.js and Bun.
C++ Format - A modern formatting library
cxx - Safe interop between Rust and C++
fast-float-rust - Super-fast float parser in Rust (now part of Rust core)
encoding_rs - A Gecko-oriented implementation of the Encoding Standard in Rust
RapidJSON - A fast JSON parser/generator for C++ with both SAX/DOM style API
bumpalo - A fast bump allocation arena for Rust
earcut.hpp - Fast, header-only polygon triangulation
feel