The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 23 Protobuf Open-Source Projects
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
SteamKit
SteamKit2 is a .NET library designed to interoperate with Valve's Steam network. It aims to provide a simple, yet extensible, interface to perform various actions on the network.
-
plumber
A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Reverse Engineering Protobuf Definitions from Compiled Binaries | news.ycombinator.com | 2024-03-09For at least 4 years protobuf has had decent support for self-describing messages (very similar to avro) as well as reflection
https://github.com/protocolbuffers/protobuf/blob/main/src/go...
Xgooglers trying to make do on the cheap will just create a Union of all their messages and include the message def in a self-describing message pattern. Super-sensitive network I/O can elide the message def (empty buffer) and any for RecordIO clone well file compression takes care of the definition.
Definitely useful to be able to dig out old defs but protobuf maintainers have surprisingly added useful features so you don’t have to.
Bonus points tho for extracting the protobuf defs that e.g. Apple bakes into their binaries.
Project mention: FlatBuffers – an efficient cross platform serialization library for many langs | news.ycombinator.com | 2023-09-18
grpcurl
Now let’s take a look at the source code of the protoc-gen-go plugin:
Project mention: 5 Open Source tools written in Golang that you should know about | dev.to | 2023-12-15The Buf CLI is a versatile tool designed for handling Protocol Buffers (Protobuf), a method of serializing structured data. It offers several key features, including managing Protobuf assets through the Buf Schema Registry (BSR), providing a linter to enforce optimal API design and structure, and a breaking change detector to maintain compatibility either in source code or at the wire level. Additionally, the Buf CLI includes a generator that activates plugins based on user-defined templates and a formatter to standardize the formatting of Protobuf files according to industry norms. It also integrates seamlessly with the Buf Schema Registry, supporting comprehensive dependency management.
The next time I want to build an API, I will probably make use of https://github.com/twitchtv/twirp. I like working with Protobuf and having strongly-typed and well-defined messages, but gRPC is way, way too much. It's obviously a Google product, built for what Google needs.
Use Protobuf for messages, but just use HTTP for transport.
Have you tried this:? https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serializers.md
Project mention: Is there a programming language that will blow my mind? | /r/ProgrammingLanguages | 2023-06-01https://github.com/asynkron/protoactor-go & this is a great lib, that implements a Erlang/Akka-like the Actor Model in Go.
Project mention: Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance | news.ycombinator.com | 2024-04-12The benchmark is not comparing apples to apples.
prost is the most widely used Protobuf implementation in Rust, maintained by the Tokio organization. prost generates structs and serialization/deserialization code for you.
easyproto according to GitHib Search is used only by two projects. easyproto provides primitives for serializing and deserializing Protobuf, and requires hand writing code to do both.
A fair comparison would be prost vs google.golang.org/protobuf, or easyproto vs parts of quick-protobuf.
In most cases you can make Go as fast as Rust, but from my experience writing performance-sensitive code in Go requires significantly larger time investment and overall requires deeper language expertise. Pebble (RocksDB replacement in Go by CockroachDB) is a good example of this, the codebase is littered with hand-inlined[1] functions, hand-unrolled loops and it's not[2] even using Go memory management for performance critical parts, it's using the C memory allocator and manual memory management.
[prost]: https://github.com/tokio-rs/prost
> C/C++'s header system with conditional inclusion
Wouldn't it be more accurate to say something like "older build systems"? I don't think any of the things you listed are "modern". Which isn't a criticism of their legacy! They have been very useful for a long time, and that's to be applauded. But they have huge problems, which is a big part of why newer systems have been created.
FWIW, I have been using pants[0] (v2) for a little under a year. We chose it after also evaluating it and bazel (but not nix, for better or worse). I think it's really really great! Also painful in some ways (as is inevitably the case with any software). And of course it's nearly impossible to entirely stomp out "genrules" use cases. But it's much easier to get much closer to true hermeticity, and I'm a big fan of that.
0: https://www.pantsbuild.org/
Project mention: DocArray – Represent, send, and store multimodal data for ML | news.ycombinator.com | 2023-04-27
So I will be making a client-side mod to show mmr changes as in the old client Upcoming data will be stored locally in the cfg folder (per account i.e. cloud synced) Wont know about matches played before adding the mod (valve went scorched earth removing data points from client.dll) But can populate previous history via an external opensource tool based on SteamKit samples
So, is JR yet another faking library written in Go? Yes and no. JR indeed implements most of the APIs in fakerjs and Go fake it, but it's also able to stream data directly to stdout, Kafka, Redis and more (Elastic and MongoDB coming). JR can talk directly to Confluent Schema Registry, manage json-schema and Avro schemas, easily maintain coherence and referential integrity. If you need more than what is OOTB in JR, you can also easily pipe your data streams to other cli tools like kcat thanks to its flexibility.
Project mention: Mastering Golang Microservices - A Practical Guide: Embrace High-Performance with Kitex and Hertz | dev.to | 2024-01-12Stay curious, keep exploring, and stay tuned for our upcoming Rust-focused Volo guide, which will introduce you to yet another exciting aspect of CloudWeGo. Happy coding!
Protobuf related posts
- Fivefold Slower Compared to Go? Optimizing Rust's Protobuf Decoding Performance
- Reverse Engineering Protobuf Definitions from Compiled Binaries
- Show HN: AuthWin – Authenticator App for Windows
- Eval("quire".replace(/^/,"re"))(moduleName)
- Google API Design Guide
- I Reviewed 1,000s of Opinions on gRPC
- Eliminating additional bandwidth charges for multi-zone sites on Vercel
-
A note from our sponsor - WorkOS
workos.com | 26 Apr 2024
Index
What are some of the best open-source Protobuf projects? This list will help you:
Project | Stars | |
---|---|---|
1 | Protobuf | 63,586 |
2 | kratos | 22,420 |
3 | FlatBuffers | 22,048 |
4 | grpcurl | 10,118 |
5 | protobuf | 9,638 |
6 | goprotobuf | 9,546 |
7 | buf | 8,232 |
8 | twirp | 6,820 |
9 | gogoprotobuf | 5,629 |
10 | kotlinx.serialization | 5,099 |
11 | protoactor-go | 4,873 |
12 | rejoiner | 3,659 |
13 | prost | 3,513 |
14 | pants | 3,100 |
15 | docarray | 2,739 |
16 | nano | 2,689 |
17 | rust-protobuf | 2,661 |
18 | protoc-gen-doc | 2,556 |
19 | SteamKit | 2,448 |
20 | rq | 2,255 |
21 | schema-registry | 2,136 |
22 | plumber | 2,041 |
23 | volo | 2,039 |
Sponsored