MessagePack VS ccache

so appreciate such a detailed reply, thanks. btw, why did you choose tinylib/msgp from 4 available go-impls?

If you find you're running the serial connection at maximum speed and it's still not fast enough, try switching to a more compact binary encoding that has both Serde and Arduino implementations, like MsgPack... though I don't remember enough about its format off the top of my head to tell you the easiest way to put an unambiguous header on each packet/message to make the protocol self-synchronizing.

The information can be stored in a database or as files, serialized in a standard format and with a schema agreed with your Data Engineering team. Depending on your information and requirements, it can be as simple as CSV, XML or JSON, or Big Data formats such as Parquet, Avro, ORC, Arrow, or message serialization formats like Protocol Buffers, FlatBuffers, MessagePack, Thrift, or Cap'n Proto.

MessagePack Similar to JSONs, just more compact, although not as much as the ones above. Still, it's usefull to retain some readability in your messages.

First of all, you're comparing this to GSON and Kryo, how does it compare to Msgpack, fast-serialization, but also Elsa and I'm sure, many others? Are there any limitations and/or trade-offs?

Upvote for msgpack, one of the great undervalued message protocols available.

I have two suggestions Capnproto, MessagePack (those are only the two examples that came to mind first, i bet there are even one or two especially developed for rust). Both of these are better than json in nearly every way.

Encode messages as JSON or MessagePack.

If you've ever inquired about MessagePack before, you probably know the phrase from its official website, msgpack.org: "It's like JSON, but fast and small." In fact, if you compare how much memory space the same data occupies in JSON and MessagePack, you'll see why the latter is a much more compact format. For example, the number 100 takes 3 bytes in JSON and only 1 in MessagePack. The difference becomes more significant as the number's order of magnitude grows. For the maximum value of int64 (9223372036854775807), the size of the stored data differs by as much as 10 bytes (19 against 9)! The same is true for boolean values---4 or 5 bytes in JSON against 1 byte in MessagePack. It is also true for arrays because many syntactic symbols---such as commas separating the elements, semicolons separating the key-value pairs, and brackets marking the array limits---don't exist in binary format. Obviously, the larger the array is, the more syntactic litter accumulates along with the payload. String values, however, are a little more complicated. If your strings don't consist entirely of quotation marks, line feeds, and other special symbols that require escaping, then you won't see a difference between their sizes in JSON and in MessagePack. For example, "foobar" has a length of 8 bytes in JSON and 7 in MessagePack. Note that the above only applies to UTF-8 strings. For binary strings, JSON's disadvantage against MessagePack is obvious.

Site oficial do MessagePack

> All those big changes introduce commits that make git bisect generally slower.

Bisection search is log2(n) so doubling the number of commits should only add one more bisection step, yes?

> Which might be awful if you also have some C code to recompile at every step of bisecting.

That reminds me, I've got to try out ccache (https://ccache.dev/ ) for my project. My full compile is one minute, but the three files that take longest to compiler rarely change.

I worked with the internals of this some 16 years ago, maintaining a customized version at Zeugma Systems. Some change of mine was reworked by someone and upstreamed:

https://github.com/ccache/ccache/commit/e8354384f67bc733bea5...

ccache https://ccache.dev/ is great for C/C++, but it would be excellent if it exposed a more generic interface for use in the way this article describes.

My first contribution to a Rust codebase was <https://github.com/mozilla/sccache/commit/da2934fcc2ed2a4ae2...>. It is adding the -fminimize-whitespace flag to the preprocessor command when available (New in Clang 14, <https://releases.llvm.org/14.0.0/tools/clang/docs/ReleaseNot...>). The equivalent ccache patch is still pending <https://github.com/ccache/ccache/pull/815>.

When using the disk cache, ccache is still faster on cache hits due to also checking the hash of all input files (called a manifest) before even executing the preprocessor. It also can just clone/hardlink the file in the cache instead of copying it.

Ccache, as the name suggests, caches compilation output, what speeds up recompilation.

I always wanted to try to use cmake-conan so I could let Conan grab all packages but have a neat cmake script being in charge of what gets built when. Also, this would allow me to easily switch between CMake fetchcontent and Conan packages that may or may not be stashed automatically on a local Artifactory server. Secondly, since now all build requirements are stashed on a server and binary reproducible, you could concider adding icecream and ccache into the mix. (Try running a node one of your buildservers for massive speadups with icecream) This however does require a reproducible build environment (by configure script) which conan again is really good in.

ccache can considerably cut down compile times. Simple to install and minimal config, no change to tooling or workflow...

https://ccache.dev/

Have u not tried ccache i just know it works with regular C/C++ . The only down side of it is that it uses some disk space. github Ps:I have never tried C# so idk if it works.