Our great sponsors
-
protobuf-es
Protocol Buffers for ECMAScript. The only JavaScript Protobuf library that is fully-compliant with Protobuf conformance tests.
-
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
-
-
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Not sure if it is a magic bullet, but it was definitely written by TypeScript developers, for TypeScript developers.
The generated TypeScript code is already pretty minimal because all serialization ops are implemented with reflection instead of generated code (which is only marginally slower than generated code in JS).
But you can also switch to generating JavaScript + TypeScript declaration files, which is truly minimal: JavaScript is an entire dynamic language, so we actually only generated a small snippet of metadata in the .js output, and create a class at run time with a function call. The generated typings (.d.ts) give you type safety, autocompletion in the IDE, and so on.
You can see the output here: https://github.com/bufbuild/protobuf-es/blob/main/packages/p...
Ye, fwiw there is an example code size comparison here:
https://github.com/bufbuild/connect-web/blob/main/packages/c...
I'm sure someone will chime in on the implementation details, but hopefully others can give it a try with their projects!
From my limited perspective, it'd certainly be nice if browsers exposed more of HTTP. I also appreciate that the web platform is far larger, and far more backward-compatible, than any code I've ever even considered writing. If browser makers are slow to expose new protocol features, that's a cost I'm happy to pay.
At least as I understand it, HTTP Push isn't the problem. The key problem is that browsers don't want to expose HTTP trailers. I understand their reluctance - even putting aside the security concerns, it's a huge semantic change. Many HTTP response headers, like etags, are typically computed by buffering the response, running some compute, then sending headers and the buffered body. All of those would, presumably, now need to also be sendable as trailers. What if the server sends both an etag header and trailer, and they don't match? The rabbit hole goes pretty far. Here's the Chromium issue where the gRPC and Chrome teams debate the issue: https://bugs.chromium.org/p/chromium/issues/detail?id=691599. There's also a series of whatwg issues about exposing trailers in the fetch API: https://github.com/whatwg/fetch/issues/34.
For a protocol that sits above L7, the gRPC HTTP/2 protocol is liberal about diving past the semantics outlined in RFC 9110 and specifying transport-level details. (As an example, trailers-only responses must be sent in a _single_ HTTP/2 headers frame with the EOS bit set.) That degree of specificity unlocks some performance gains, but it makes it near-impossible to speak this protocol using an existing HTTP library - a traditional strength of building on top of L7. For broad adoption in a _lot_ of environments, gRPC-Web (and Connect) are IMO much better protocols.
Take a look at Twirp (https://github.com/twitchtv/twirp) open sourced by TwitchTv. It's a lot lighter weight than gRPC. It does use Protobufs but addresses some of the concerns you mentioned, such as being able to test with JSON payloads, works over HTTP 1.1 and HTTP/2, good client libraries, and doesn't require a proxy.
They address your concerns in more detail in the Twirp release announcement (2018) - https://blog.twitch.tv/en/2018/01/16/twirp-a-sweet-new-rpc-f...
For those griping about using JSON manually with their APIs ... This tool is a big help for working with GRPC (at least from cli)
https://github.com/fullstorydev/grpcurl
There’s also BloomRPC, which is like Postman/Insomnia for GRPC:
https://github.com/bloomrpc/bloomrpc
And if you’re interested in consuming Twirp from a browser, I wrote TwirpScript: https://github.com/tatethurston/twirpscript
It makes sense for some use cases but the vast majority of use cases, parsing overhead is simply not a concern. Mobile phones are fast, networks have plenty of bandwidth (and the savings are marginal), parsers are pretty good.
But done right, binary protocols are sometimes worth the marginal savings they provide. We switched over one of our APIs to use CBOR instead of json. It's a search API that we hit a lot and I wanted to cut down on the bytesize of the responses a little. The savings are not that impressive. But I'll take 10% when i can get it.
Otherwise, this was a pretty simple change. We use kotlinx serialization in a multi-platform library. Basically, all we did is configure it to use CBOR instead of json. https://github.com/Kotlin/kotlinx.serialization/blob/master/... Half hour job. Haven't looked at it since; just works. It supports protobuf as well but it looked like more hassle to set up so we went with CBOR instead.