sonic VS simdjson

Also worth looking at https://github.com/bytedance/sonic

The article inside does not mention this.

Good morning.I hope this is not the wrong place to post… so let me introduce my first public Golang package. This is a JSON decoder called Sonnet ( https://github.com/sugawarayuuta/sonnet ) that has given faster results (at least in my environment) than Sonic - https://github.com/bytedance/sonic (which is said that it's the fastest) without the help of assembly!

JSON is a very well-known file format. It is used by everyone who does programming. However, it is also not uncommon to find problems with encoding/json and other third-party libraries. for more… (see https://github.com/sugawarayuuta/sonnet#problems-we-had )

I decided to create a new, standard library-compatible decoder that would be both easy to use and fast.

Thanks for reading, feel free to use, help, or ask questions, I look forward to hearing from you. All benchmarks and other information can be found in the link at the top.

Good morning. Let me introduce my first public Go package. This is a JSON decoder called Sonnet ( https://github.com/sugawarayuuta/sonnet ) that has given faster results (at least in my environment) than Sonic (which is said that it's the fastest) without the help of assembly!

I'd say the https://github.com/bytedance/sonic has the fastest encoder due to C and assembly optimization. (Use at your own risk.)

Also see sonic for some nice performance tricks github.com/bytedance/sonic

I believe https://github.com/bytedance/sonic currently has the fastest unmarshalling performance.

It is mentioned here: https://github.com/bytedance/sonic/blob/a577eafc253adb943924..., but it isn't included in the benchmarks graphs. Seems this repo is specifically focused on Golang and isn't necessarily motivated by being the fastest JSON [de]serializer on the planet.

Everything you said is totally reasonable. I'm a big fan of napkin math and theoretical upper bounds on performance.

simdjson (https://github.com/simdjson/simdjson) claims to fully parse JSON on the order of 3 GB/sec. Which is faster than OP's Go whitespace parsing! These tests are running on different hardware so it's not apples-to-apples.

The phrase "cannot go faster than this" is just begging for a "well ackshully". Which I hate to do. But the fact that there is an existence proof of Problem A running faster in C++ SIMD than OP's Probably B scalar Go is quite interesting and worth calling out imho. But I admit it doesn't change the rest of the post.

The walkthrough is very nice, how to do this if you're going to do it.

If you're going for pure performance in a production environment you might take a look at Daniel Lemire's work: https://github.com/simdjson/simdjson. Or the MinIO port of it to Go: https://github.com/minio/simdjson-go.

I have same question as /u/JDRiverRun : how do you deal with JSON, do you parse json on Rust side or on Emacs side. I see that you are requiring json.el in your lspce.el, but I haven't looked through entire file carefully. If you parse on Rust side, do you use simdjson (there are at least two Rust bindings to it)? If yes, what are your impressions, experiences compared to more "standard" json library?

I expect a lot of transcoders will be rewritten when the Vector instructions land. You can see speedups when used in other languages, such as simdjson. Please try to be more thoughtful and not disregard other people's hard work so easily.

You will never make 150GB/s though. Not even close. For example, simdjson achieves 3GB/s and is by far the fastest JSON parser out there.

Would you consider SIMD operations parallel? https://github.com/simdjson/simdjson

I'm talking about with simdjson. Lemire suggested reading line-by-line is not a good idea [0]. So I'm asking about the ideal approach using simdjson, not JSON parsers in general.

[0] https://github.com/simdjson/simdjson/issues/188#issuecomment...

Hey folks, if you've been in IRC this last week you've probably heard me mumbling to myself, or crying out in agony trying to bind Raku to simdjson. However, the pain was not in vain!