Wire VS Protobuf

It is long over due. I used Wire for my previous projet to generate Kotlin files from proto.

> didn’t find any standard for separating protobuf messages

The fact that protobufs are not self-delimiting is an endless source of frustration, but I know of 2 standards:

- SerializeDelimited* is part of the protobuf library: https://github.com/protocolbuffers/protobuf/blob/main/src/go...

- Riegeli is "a file format for storing a sequence of string records, typically serialized protocol buffers. It supports dense compression, fast decoding, seeking, detection and optional skipping of data corruption, filtering of proto message fields for even faster decoding, and parallel encoding": https://github.com/google/riegeli

I actually went through all projects listed in [1] because I remember this very quirk. It turns out that there are many such libraries that have two variants of encode/decode functions, where the second variant prepends a varint length. In my brief inspection there do exist a few libraries with only the second variant (e.g. Rust quick-protobuf), which is legitimately problematic [2].

But if the project in question was indeed protobuf.js (see loeg's comments), it clearly distinguishes encode/decode vs. encodeDelimited/decodeDelimited. So I believe the project should not be blamed, and the better question would be why so many people chose to add this exact helper. Well, because Google itself also had the same helper [3]! So at this point protobuf should just standardize this simple framing format (with an explicitly different name though), instead of claiming that protobuf has no obligation to define one.

[1] https://github.com/protocolbuffers/protobuf/blob/main/docs/t...

[2] https://github.com/tafia/quick-protobuf/issues/130

[3] https://protobuf.dev/reference/java/api-docs/com/google/prot...

[4] https://github.com/protocolbuffers/protobuf/blob/main/src/go...

It looks like it is in fact universal. Just glancing at the code here, it looks like the tool searches any arbitrary file for bytes that look like encoded protobuf descriptors, specifically looking for bytes that are plausibly the beginning of a FileDescriptorProto message defined here:

https://github.com/protocolbuffers/protobuf/blob/main/src/go...

This takes advantage of the fact that such descriptors are commonly compiled into programs that use protobuf. The descriptors are usually embedded as constant byte arrays. That said, not all protobuf implementations embed the descriptors and those that do often have an option to inhibit such embedding (at the expense of losing some dynamic introspection features).

On optional.. this was a regression in proto that is somewhat helped by https://github.com/protocolbuffers/protobuf/blob/main/docs/f... ; I have no idea whether protobuf for rust has started taking advantage of this.

JSON is awful in every way.

recent versions of proto3 have added back the “optional” keyword that can be used on any field. see: https://github.com/protocolbuffers/protobuf/blob/main/docs/f...

The protobuf compiler installed on your local computer. We will use this to generate the language-specific stubs from our data format.

Check out protocol buffers https://protobuf.dev/overview/#benefits https://protobuf.dev/ https://github.com/protocolbuffers/protobuf

#!/bin/sh echo "Installing Rust and Wasm toolchain..." curl https://sh.rustup.rs -sSf | sh -s -- -y rustup default stable rustup update rustup update nightly rustup target add wasm32-unknown-unknown --toolchain nightly echo "Installing protoc..." sudo apt update sudo apt install -y unzip PROTOC_VERSION=$(curl -s "https://api.github.com/repos/protocolbuffers/protobuf/releases/latest" | grep -Po '"tag_name": "v\K[0-9.]+') curl -Lo protoc.zip "https://github.com/protocolbuffers/protobuf/releases/latest/download/protoc-${PROTOC_VERSION}-linux-x86_64.zip" sudo unzip -q protoc.zip bin/protoc -d /usr/local sudo chmod a+x /usr/local/bin/protoc rm -rf protoc.zip echo "Building Substrate Node Template..." cargo build --release --locked echo "Installing cargo-contract CLI tool..." cargo install --force --locked cargo-contract --version 2.0.0-rc

gRPC uses protocol buffers (it is an open source message format) as the default method of communication between client and server. Also, gRPC uses HTTP/ 2 as the default protocol.