protobuf
eRPC
protobuf | eRPC | |
---|---|---|
2 | 2 | |
48,001 | 824 | |
- | 1.0% | |
9.7 | 4.6 | |
almost 3 years ago | 7 days ago | |
C++ | C++ | |
GNU General Public License v3.0 or later | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
protobuf
-
Help in explaining the reason for an error [R]
=> ERROR [11/14] RUN curl -OL "https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip" && 1.6s
-
gRPC Hola mundo con golang
# Make sure you grab the latest version curl -OL https://github.com/google/protobuf/releases/download/v3.5.1/protoc-3.5.1-linux-x86_64.zip # Unzip unzip protoc-3.5.1-linux-x86_64.zip -d protoc3 # Move protoc to /usr/local/bin/ sudo mv protoc3/bin/* /usr/local/bin/ # Move protoc3/include to /usr/local/include/ sudo mv protoc3/include/* /usr/local/include/ # Optional: change owner sudo chown [user] /usr/local/bin/protoc sudo chown -R [user] /usr/local/include/google
eRPC
-
Are You Sure You Want to Use MMAP in Your Database Management System?
The most common example is DPDK [1]. It's a framework for building bespoke networking stacks that are usable from userspace, without involving the kernel.
You'll find DPDK mentioned a lot in the networking/HPC/data center literature. An example of a backend framework that uses DPDK is the seastar framework [2]. Also, I recently stumbled upon a paper for efficient RPC networks in data centers [3].
If you want to learn more, the p99 conference by ScyllaDB has tons of speakers talking about some interesting challenges.
[1] https://www.dpdk.org/.
[2] https://github.com/scylladb/seastar
[3] https://github.com/erpc-io/eRPC
-
Zero-copy network transmission with io_uring
My side project has been to rewrite https://github.com/erpc-io/eRPC which does RPCs over UDP with some congestion control supposedly quite fast. Paper: https://www.usenix.org/system/files/nsdi19-kalia.pdf. I never got the code to work in AWS; I believe the author focused on Mellanox NICS but that's not really commodity H/W, which is where my interests lay.
So I dug into it .. and well I'll have my own library soon. I should be able to send UDP w/o congestion control sometime this week.
eRPC uses DPDK (100% user space NIC TX/RX control) plus the author's own other ideas to get performance. Since I'm getting into NICs + DPDK (in a serious way i.e this much more involved than vanilla sys/socket.h I/O) way, way late in the game, I hope and believe DPDK is, in the medium term, the better way to go than to turn to kernel improvements in for I/O.
Like others, getting the kernel out of the way, with pinned threads seems cleaner if one can develop from scratch.
This library will be a part of something bigger, however, a key architecture point for many people is: I got a RPC/packet/message. Now should I:
* process it in-place e.g. on the thread that was doing RX?
* delegate it to another core?
* if I delegate .. to whom?
* If I delegate how do I get a response back?
In DPDK I believe these are easier to decide and well-manage in code.
What are some alternatives?
erpc - Embedded RPC
liburing
protox - A fast, easy to use and 100% conformant Elixir library for Google Protocol Buffers (aka protobuf)
bakelite - Bakelite is a utility that makes it simple to communicate with your firmware.
buf - The best way of working with Protocol Buffers.
ice - All-in-one solution for creating networked applications with RPC, pub/sub, server deployment, and more.
nanopb-example - This is a simple project created to test the capabilities of Google's protobuf C implementation, nanopb.
netty-incubator-transport-io_uring
protobuf - A pure Elixir implementation of Google Protobuf.
FlatBuffers - FlatBuffers: Memory Efficient Serialization Library
buf-examples - Example repository that uses Buf.
areg-sdk - AREG is an asynchronous Object RPC framework to simplify multitasking programming by blurring borders between processes and treating remote objects as if they coexist in the same thread.