NFS v4.2 and server-side copying (and how I got it to work)

This page summarizes the projects mentioned and recommended in the original post on /r/linux

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • Homeserver

    Lightweight home server based on microservices, usable as desktop workstation

  • I have a Linux NFS server and clients (all are running Linux kernel 5.13 or higher) and I was able to set up network mounts by using the NFS version 4.2 (by using this guide here: https://github.com/zilexa/Homeserver/tree/master/filesystem/networkshares_HowTo-NFSv4.2). And the reason why I wanted NFSv4.2 was server-side copying feature, see: https://superuser.com/questions/337778/copy-remote-files-on-nfs-without-round-trip I think it's a killer feature, like reflinking over NFS (which works with BTRFS and XFS filesystems over NFS too!).

  • coreutils

    upstream mirror (by coreutils)

  • But even I had successfully set up the NFS v4.2, my copying tests (cp /mnt/nfs/src.file /mnt/nfs/dest.file) were indicating that the data is still round-tripping via the client. Then I was informed that it's not enough that shares are using NFSv4.2: the user-space tool that is used to copy the files need to support "copy_file_range" syscall as well, to get the server-side copying to work properly. And it seems that this feature has been recently added to coreutils' cp command (see: https://github.com/coreutils/coreutils/commit/4b04a0c3b792d27909670a81d21f2a3b3e0ea563) and it's available on coreutils v9.0 onwards. Unfortunately, Ubuntu's and Debian's coreutils are still on v8.xx and they lack this functionality. So I compiled manually coreutils 9.0 on one of my Debian 11 NFS client machine and it works! Now if I copy something inside the NFS share (or from another NFS share to another, which are mounted from the same NFS server) the data gets copied on the NFS server locally and it's not round-tripping via the client anymore. This is a significant performance gain in certain circumstances. This feature can still be easily disabled by using --reflink=never argument with cp command (in coreutils 9.0 cp command defaults to --reflink=auto: https://www.reddit.com/r/linux/comments/punk0d/gnu_coreutils_90/he3yqoj/).

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • speedcopy

    Patched python shutil.copyfile to allow faster speeds on samba shares.

  • Interestingly there is a python module which does support that functionality: https://github.com/antirotor/speedcopy

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts