Code-used-on-Daniel-Lemire-s-blog VS miri

It's not bad at all https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/...

A newline is missing in the example code. As given there's a segfault at line 5 not line 6.

However, the code at https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/... shows it's indeed at line 6, because it has an extra newline after the '#include '.

Daniel Lemire's Blog : Daniel Lemire's blog

Dr. Daniel Lemire's blog: https://lemire.me/blog, covers lots of technical items on optimizations in various programming languages, Lemire's work is currently in use across a number of projects and he consistently delivers fantastic improvements, he usually accompanies these improvements with a blog post describing what he did. He also occasionally posts interesting Science and Technology links on various topics not limited to tech, but health and education as well.

Nope, simdjson is originally from Daniel Lemire who also often blogs about fancy low level optimizations: https://lemire.me/blog/ I'm just a happy user :)

Added this pull request with some interesting results.

There's a copy of the loop used on the escape function inside the avx512_escape function [0]. Is it needed or just a copy and paste mistake? (I know nothing about vector instructions)

0: https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/...

Provenance is a dynamic property of pointer values. The actual underlying rules that a program must follow, even when using raw pointers and `unsafe`, are written in terms of provenance. Miri (https://github.com/rust-lang/miri) represents provenance as an actual value stored alongside each pointer's address, so it can check for violations of these rules.

Lifetimes are a static approximation of provenance. They are erased after being validated by the borrow checker, and do not exist in Miri or have any impact on what transformations the optimizer may perform. In other words, the provenance rules allow a superset of what the borrow checker allows.

There has been discussion of doing this with MIRI, which would be easier than all of rustc.

This issue has been fixed: https://github.com/rust-lang/miri/issues/2964

Actually, I've done more advanced tests with MIRI (see https://github.com/rust-lang/miri/issues/2920 for example) which allowed me to fix some issues. I've also made the code compatible with loom, but I didn't found the time yet to write and execute loom tests. That's on the TODO-list, and I need to track it with an issue too.

He is one of the big brains behind Miri, which is a interpreter that runs on the MIR (compiler representation between human code and asm/machine code) and detects undefined behavior. Super useful tool for language safety, pretty interesting on its own.

I would also run your tests in Miri (https://github.com/rust-lang/miri) to try to cover more bases.

You can run miri and it will tell you if the given run triggered any undefined behavior. It will not analyze it for every possible use of the code, but checking for the presence of this specific issue using it should be fairly simple.

If you do encounter a piece of code on which TB performs much worse than SB, do submit it as an issue! There was one recently and we massively improved TB performance on this case by improving garbage collection.