unix-history-repo VS Cap'n Proto

There is also https://github.com/dspinellis/unix-history-repo (Continuous Unix commit history from 1970 until today)

FWIW, ls in Research-V6 back in 1975 had 10 options. https://github.com/dspinellis/unix-history-repo/blob/Researc...

By BSD 3 in 1980 it had 11 options. https://github.com/dspinellis/unix-history-repo/blob/BSD-3-S...

The thing is, we can see even from the 1970s 'ls' how the Unix model doesn't meet the goal "to chain these simple programs together to create complex behaviors".

There is no option to escape or NUL terminate a filename, making it possible to construct a filename containing a newline which makes the output look like two file entries.

The option for that was added later.

There's also the issue that embedded terminal codes will be interpreted by the terminal.

This is what it looked like about 7-8 years earlier: https://github.com/dspinellis/unix-history-repo/blob/BSD-1/e...

RA92 (1989): 16 ms / 8.3 ms.

Note that the RL02 (and V7) and RA92 mentioned in the article are separated by about a decade.

[1] https://github.com/dspinellis/unix-history-repo/blob/Researc...

The earliest version I could find [1] is already written in C.

[1] https://github.com/dspinellis/unix-history-repo/blob/Researc...

Here's a 1997 citation for "top cpu processes." It's not as close to the original 1984 release as I would like, but it's better than Wikipedia. https://github.com/dspinellis/unix-history-repo/commit/aee34003d7964653c44c31f5bf6bcf136b32c4f3

> The “why” goes into the PR and more importantly, engineering documentation and inline comments

This just ensures that the “why” is lost when someone comes looking years later.

From experience, SCM metadata is far more durable than just about any other work product we produce. Five decades later and RCS commit info was still available for the Unix sources, and history could be reconstructed: https://github.com/dspinellis/unix-history-repo

I’ve used 35-year-old commit messages to help understand a long-standing issue, decades after all other related organization tooling and data had disappeared.

P.S. I remember I looked into early versions of C (they survived in Unix historic releases) and that, finally, revealed to me why C does something really stupid and conflates arrays and slices (pointers). Initially C had no arrays! Or, rather, what it called arrays were, actually, pointers. “Normal” arrays were added at some point, but because these weird slices/pointers were already there that caused endless confusion. It wasn't resolved before C became popular and after that it was too late. Go repeated that mistake with slices, of course.

Yeah I pretty much only use my own alternate container implementations (from KJ[0]), which avoid these footguns, but the result is everyone complains our project is written in Kenton-Language rather than C++ and there's no Stack Overflow for it and we can't hire engineers who know how to write it... oops.

[0] https://github.com/capnproto/capnproto/blob/v2/kjdoc/tour.md

- may massively reduce the latency involved.

Those sharing Cap'n Proto-encoded data may have particular interest. Cap'n Proto (https://capnproto.org) is fantastic at its core task - in-place serialization with zero-copy - and we wanted to make the IPC (inter-process communication) involving capnp-serialized messages be zero-copy, end-to-end.

That said, we paid equal attention to other varieties of payload; it's not limited to capnp-encoded messages. For example there is painless (<-- I hope!) zero-copy transmission of arbitrary combinations of STL-compliant native C++ data structures.

To help determine whether Flow-IPC is relevant to you we wrote an intro blog post. It works through an example, summarizes the available features, and has some performance results. https://www.linode.com/blog/open-source/flow-ipc-introductio...

Of course there's nothing wrong with going straight to the GitHub link and getting into the README and docs.

Currently Flow-IPC is for Linux. (macOS/ARM64 and Windows support could follow soon, depending on demand/contributions.)

FWIW, my C++ toolkit library, KJ, does the same thing.[0]

But presumably you could still write a condition predicate which looks at things which aren't actually part of the mutex-wrapped structure? Or does is the Rust type system able to enforce that the callback can only consider the mutex-wrapped value and values that are constant over the lifetime of the wait? (You need the latter e.g. if you are waiting for the mutex-wrapped value to compare equal to some local variable...)

[0] https://github.com/capnproto/capnproto/blob/e6ad6f919aeb381b...

The second one is to encode data in such a way that you can read it and operate on it directly from the buffer. You write data in a layout that is the same, or easily transformed as types in memory. To do that you usually need to encode with a known schema, only Sized types to efficiently compute fields locations as offsets in the buffer, and you usually represent pointers as offset into the encode. You can look at capnproto protocol for instance https://capnproto.org/

Worked well for Cap'n Proto (the cerealization protocol)! https://capnproto.org/

With all due respect, you read completely wrong.

* The very first use case for which Cap'n Proto was designed was to be the protocol that Sandstorm.io used to talk between sandbox and supervisor -- an explicitly adversarial security scenario.

* The documentation explicitly calls out how implementations should manage resource exhaustion problems like deep recursion depth (stack overflow risk).

* The implementation has been fuzz-tested multiple ways, including as part of Google's oss-fuzz.

* When there are security bugs, I issue advisories like this:

https://github.com/capnproto/capnproto/tree/v2/security-advi...

* The primary aim of the entire project is to be a Capability-Based Security RPC protocol.

I like how they use capability-based security [0] and use Cap'n Proto protocol. This is another technology that is slow to get broad adoption, but has many things going for when compared to e.g. Protocol Buffers (Cap'n Proto is created by the primary author of Protobuf v2, Kenton Varda).

[0] https://sandstorm.io/how-it-works#capabilities

[1] https://capnproto.org

Related but not Rust-specific: FlatBuffers, Cap'n Proto.