Making Rust Float Parsing Fast: libcore Edition

• rust

  2,692 94,153 10.0 Rust

  Empowering everyone to build reliable and efficient software.

  

A little over 2 years ago, I posted about making Rust float parsing fast and correct. Due to personal reasons, I needed a break prior to merging improvements into the Rust core library. In that time, major improvements to float parsing have been developed, further improving upon anything I had done. And, as of today, these changes have been merged into the core library.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• fast-float-rust

  10 271 0.0 Rust

  Super-fast float parser in Rust (now part of Rust core)

JSON is an interesting example, since the syntax of valid floats is slightly different than what Rust expects. Luckily, I'm the author of a minimal, and am also the author of a PR to bring this to fast-float-rust.

• fast_float

  15 1,296 8.5 C++

  Fast and exact implementation of the C++ from_chars functions for number types: 4x to 10x faster than strtod, part of GCC 12 and WebKit/Safari

  

Daniel Lemire @lemire (creator of the algorithm, author of the C++ implementation, and provided constant feedback to help guide the PR).

• fast-float-asm

  1 0 1.8 Rust

  Utilities to check AMS generation for fast-float-rust

So, what about safety? fast-float-rust uses unsafe code fairly liberally, so how do the merged changes fix that? Well, all unsafe code except when needed was removed, and it was shown to have no impact on performance or even assembly generation. In fact, the generated assembly is slightly better in some cases. Every call to an unsafe function can be trivially shown to be correct, and all but 1 call has the following format:

• rust-lexical

  2 285 0.0 Rust

  Fast numeric to- and from-string conversion routines.

If you're interested, I've basically compiled a list of all the different float formats from over 50 programming and data interchange formats.

• simdutf8

  15 512 5.9 Rust

  SIMD-accelerated UTF-8 validation for Rust.

  

No, libcore uses simple branching code at the moment, see https://github.com/rust-lang/rust/issues/68455. The issue is still actively being worked on. Note, it's not a simple drop in, and there seem to be even faster algorithms. For now there is https://github.com/rusticstuff/simdutf8.

Suggest a related project