MessagePack VS ck

so appreciate such a detailed reply, thanks. btw, why did you choose tinylib/msgp from 4 available go-impls?

If you find you're running the serial connection at maximum speed and it's still not fast enough, try switching to a more compact binary encoding that has both Serde and Arduino implementations, like MsgPack... though I don't remember enough about its format off the top of my head to tell you the easiest way to put an unambiguous header on each packet/message to make the protocol self-synchronizing.

The information can be stored in a database or as files, serialized in a standard format and with a schema agreed with your Data Engineering team. Depending on your information and requirements, it can be as simple as CSV, XML or JSON, or Big Data formats such as Parquet, Avro, ORC, Arrow, or message serialization formats like Protocol Buffers, FlatBuffers, MessagePack, Thrift, or Cap'n Proto.

MessagePack Similar to JSONs, just more compact, although not as much as the ones above. Still, it's usefull to retain some readability in your messages.

I'll try that msgpack.org website.

First of all, you're comparing this to GSON and Kryo, how does it compare to Msgpack, fast-serialization, but also Elsa and I'm sure, many others? Are there any limitations and/or trade-offs?

Upvote for msgpack, one of the great undervalued message protocols available.

I have two suggestions Capnproto, MessagePack (those are only the two examples that came to mind first, i bet there are even one or two especially developed for rust). Both of these are better than json in nearly every way.

Encode messages as JSON or MessagePack.

Maybe I'm missing something, but x is not volatile and the compiler is free to assume that it is not modified concurrently outside the bounds of C's memory model. Compilers can and do hoist out loop invariants, and https://github.com/concurrencykit/ck/commit/b54ae5c4ace9b94442bbb46858449069f566d269 seems like an example of compilers doing what you say they don't. What am I missing?

Recommend checking out http://concurrencykit.org instead.

If you're interested in how other approaches work, or how one achieves concurrency on shared mutable state without mutual exclusion, would recommend checking out concurrency kit.

There are plenty of practical solutions to the safe memory reclamation problem in C. The language just doesn't force one on you.

From epoch-based reclamation (https://github.com/concurrencykit/ck/blob/master/include/ck_..., especially with the multiplexing extension to Fraser's classic scheme), to quiescence schemes (https://liburcu.org/), or hazard pointers (https://github.com/facebook/folly/blob/master/folly/synchron..., or https://pvk.ca/Blog/2020/07/07/flatter-wait-free-hazard-poin...)... or even simple using a type-stable (https://www.usenix.org/legacy/publications/library/proceedin...) memory allocator.

In my experience, it's easier to write code that is resilient to hiccups in C than in Java. Solving SMR with GC only offers something close to lock-freedom when you can guarantee global GC pauses are short enough... and common techniques to bound pauses, like explicitly managed freelists land you back in the same problem space as C.

ck - Concurrency primitives, safe memory reclamation mechanisms and non-blocking data structures. BSD-2-Clause

Indeed they do, https://github.com/concurrencykit/ck