bita
cdc-file-transfer
Our great sponsors
bita | cdc-file-transfer | |
---|---|---|
3 | 23 | |
212 | 2,718 | |
- | 2.9% | |
3.3 | 10.0 | |
4 days ago | 18 days ago | |
Rust | C++ | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
bita
-
CDC File Transfer
Built this cdc tool for software update of embedded (Linux) systems and have deployed it with good enough performance on a couple of arm CPUs; https://github.com/oll3/bita
Though main goal has been keeping data usage low rather than speed up.
-
rsync, article 3: How does rsync work?
Nice write up. rsync is great as an application but I found it more cumbersome to use when wanting to integrate it into my own application. There's librsync but the documentation is threadbare and it requires an rsync server to run. I found bita/bitar (https://github.com/oll3/bita) which is inspired by rsync & family. It works more like zsync which leverages HTTP Range requests so it doesn't require anything running on the server to get chunks. Works like a treat using s3/b2 storage to serve files and get incremental differential updates on the client side!
-
KySync: A complete modern C++ rewrite of Zsync with 3x-10x+ performance boost
Very cool, thanks for sharing. I did a deep dive in the past into various syncing/binary diff protocols and really liked zsync. It was probably my top choice for the application I was designing but I ended up not using it. The library I did use is called bita: https://github.com/oll3/bita. It is inspired by the same family of projects as zsync. The main advantage I found with bita is that the core logic is encapsulated in a library so that you don’t only have to use the binaries but can integrate it directly into an application. I’d be curious to know if that’s in the plans for KySync.
cdc-file-transfer
-
Born from the ashes of Stadia, this repository contains tools for synching and streaming files from Windows to Linux.
The README has pretty good explanation of why it's better than rsync, and the animations help show exactly what the difference is.
- Google made a tool like rsync which is 3x faster
-
CDC File Transfer
Slightly OT, but I like the schematic gifs used in the Readme.md (pretty amazing doc overall!) like this one [0]. Does anyone have suggestions what tools they might have used (or might be used in general) to create those?
[0] https://github.com/google/cdc-file-transfer/blob/main/docs/l...
The documentation in the code itself is pretty great as well:
https://github.com/google/cdc-file-transfer/blob/main/fastcd...
The same question was asked here: https://github.com/google/cdc-file-transfer/issues/56
We also ran the experiment with the native Linux rsync, i.e syncing Linux to Linux, to rule out issues with Cygwin. Linux rsync performed on average 35% worse than Cygwin rsync, which can be attributed to CPU differences.
Windows to Windows is being worked on, see https://github.com/google/cdc-file-transfer/compare/main...s....
Linux to Linux is also an option if there is demand, but currently it's Windows to Linux only.
What are some alternatives?
yaydl - yet another youtube down loader (Git mirror)
humble-cli - 📦 The missing CLI for downloading your Humble Bundle purchases
d2 - D2 is a modern diagram scripting language that turns text to diagrams.
imsy - simple incremental pull of immutable large files
async-subscription-map - Async bookkeeping datastructure for dynamic state subscriptions across tasks
Killed by Google - Part guillotine, part graveyard for Google's doomed apps, services, and hardware.
got - Got is like git, but with an 'o'
swupd-client - Software update client