Ask HN: Best modern file transfer/synchronization protocol?

• rclone

  963 43,840 9.8 Go

  "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files

  

seconding rsync and syncthing.

the server could expose an smb or nfs share, the client could mount it, and then sync to that mount.

rsync over ssh also works, if you do not want to run smb/nfs.

this is also a cool tool https://rclone.org/

• tusd

  15 2,863 9.1 Go

  Reference server implementation in Go of tus: the open protocol for resumable file uploads

  

If it’s one way (that wasn’t quite clear from the requirements to me).

take a look at https://tus.io/

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• syncthing-android

  33 1,059 8.8 Java

  Syncthing-Fork - A Syncthing Wrapper for Android. (by Catfriend1)

I built something on top of the Syncthing API this week after using it on its own for years.

A local instance of Syncthing can behave as a robust sync tool + inotify API for applications consuming the files: https://docs.syncthing.net/rest/events-get.html#get-rest-eve...

i believe there's an embeddable golang library, but if you want something easy to use on android check in on syncthing-fork which lets you define more granular sync conditions including "just turn on 5 minutes every hour" https://github.com/Catfriend1/syncthing-android

• unison

  27 3,726 8.5 OCaml

  Unison file synchronizer (by bcpierce00)

I highly recommend Unison (https://github.com/bcpierce00/unison)

It allows you to sync between 2 machines (bi-directional) over TCP or SSH.

Note that TCP way is not encrypted, you may use wireguard as transport layer encryption for that purpose...

You can use an external application to copy if file size is larger than an arbitrary number. (Eg: use rsync for files > 1gb)

• magic-wormhole

  74 18,180 8.1 Python

  get things from one computer to another, safely

  

On a similar note, can someone tell me what's the fastest (wireless) way to transfer files between two laptops on same network (i.e. hotspot)?

scp, rsync, wormhole give me only 2-3 mb/s.

What's causing the bottleneck here and what am I missing? Thanks in advance!

https://github.com/magic-wormhole/magic-wormhole

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project