zfs VS hfs

https://openzfsonosx.org is a great starting place for general ZFS on Mac stuff.

I am imaging the following simple situation. I am using open ZFS on macOS (OpenZFS on OS X), and I have created a zpool with a single external USB drive. What command should I run when the drive is accidentally disconnected without properly exporting the pool?

This is completely wrong. ZFS works very well via O3X [0]. I ran my home folder off of ZFS on various Mac Pros for ~11 years. ZFS works well in Linux and FreeBSD as well.

----

0: https://openzfsonosx.org/

This might be able to read the drive https://openzfsonosx.org/ But I don't know if Time Machine will jive with it.

After dabbling with early stuff from the sadly scrapped Apple effort, I started my full time Mac ZFS journey in 2011 with the then Z-410, which then became ZEVO which was 10.8 only, and then staying on that (and in turn 10.8 doho) on a Mac Pro 3,1 for a solid 4 years or so. I was really sad when that effort didn't pan out and the company got acquired and then the whole deal was killed, but O3X revived the torch and I moved directly onto that. I've carried my pool forward continuously the whole way, with snapshots and everything, all data scrubbed and known good with no more data rot. It's a damn shame events (software patents) conspired to keep that from becoming a universal native fs, but it's still a wonderful thing and Lundman's amazing consistent efforts have been fantastic. The main site at https://openzfsonosx.org/ has decently active forums for something niche.

Without Apple on board or a much bigger effort ZFS on the Mac will probably always have some real limitations particularly with the GUI. For one of my desktop systems I've migrated to running my home and data folders as HFS or APFS formatted iSCSI targets, so it's still on ZFS underneath but appears native to macOS (and that also means that the effective death of towers isn't a limit on storage). But this remains an incredible project IMO.

Also for anyone who wants some limited GUI interaction capability, there is a small project called ZetaWatch [0] which will put some ZFS control into a menubar widget.

----

0: https://github.com/cbreak-black/ZetaWatch/

OpenZFS is available for MacOS and it's about as rock solid as you get in a file system. As a bonus, if you someday need to migrate the dataset to a server, you'll be able to import the zfs pool pretty easily.

Check out https://openzfsonosx.org/

Site ZFS on OSX seems down for the last few days...

>such as ZFS

FWIW, ZFS has actually been available on macOS for a long time in various iterations (MacZFS, Z-410/ZEVO, and now OpenZFS [0]). I ran it for around 11 years on a number of systems with almost all of my data on it (home folder, applications, /opt for MacPorts etc) and it was a tank. Fun project and well worth checking out. I'll always regret the various factors that meant Apple didn't adopt it as their native FS.

----

0: https://openzfsonosx.org/

I believe most of the open source HFS code was from NeXT rather than Mac OS. A lot of classic Mac OS was still 68k assembly.

eg:

https://github.com/apple-oss-distributions/hfs/blob/hfs-106/...

13-Jan-1998 jwc first cut (derived from old NextStep macfs.util code and cdrom.util code).

I just looked into this, since what you say and what Apple’s documentation says are two different things.

Here is Apple’s documentation:

https://devstreaming-cdn.apple.com/videos/wwdc/2019/419ef9ip...

F_BARRIERFSYNC: fsync() with a barrier

F_FULLFSYNC: Drive flush its cache to disk

This sounds like the Linux fsync() and Linux syncfs() respectively. What you say is that F_FULLFSYNC is the same as Linux fsync() and your performance numbers back that up. Unfortunately, you would only see a difference between Linux fsync() and Linux syncfs() if you have files being asynchronously written at the same time as the files that are subject to fsync()/syncfs(). fsync() would only touch the chosen files while syncfs() would touch both. If you did not have heavy background file writes and F_FULLSYNC really is equivalent to syncfs(), you would not be able to tell the difference in your tests.

That said, let’s look at how this actually works on Mac OS. Unfortunately, the apfs driver does not appear to be open source, but the HFS+ driver is. Here are the relevant pieces of code in HFS+:

https://github.com/apple-oss-distributions/hfs/blob/hfs-556....

https://github.com/apple-oss-distributions/hfs/blob/5e3008b6...

First, let me start with saying this merits a faceplam. The fsync() operation is operating at the level of the mount point, not the individual file. F_FULLSYNC and F_BARRIERFSYNC are different, but they both might as well be variants of the Linux syncfs().

For good measure, let us look at how this is done on the MacOS ZFS driver:

https://github.com/openzfsonosx/zfs/blob/master/module/zfs/z...

The file is properly synced independently of the mountpoint, such that other files being modified in the file are not immediately required to be written out to disk. That said, both F_FULLSYNC and F_BARRIERFSYNC on MacOS are mapped by the ZFS driver to the same function that implements fsync() on Linux:

https://github.com/openzfs/zfs/blob/master/module/os/linux/z...

For good measure, let us look at how syncfs() is implemented by ZFS on Linux:

https://github.com/openzfs/zfs/blob/master/module/os/linux/z...

It operates on the superblock, which is what MacOS’ HFS+ driver does.

From this, I can conclude:

Linux syncfs() == macOS F_FULLFSYNC on HFS+

Linux fsync() == macOS fsync()/F_FULLFSYNC/F_BARRIERFSYNC on ZFS

Also, MacOS F_BARRIERSYNC is a weakened Linux syncfs() and Apple’s documentation is very misleading (although maybe not technically wrong). POSIX does allow fsync to be implemented via syncfs (sync in POSIX, but I am saying syncfs from Linux to be less confusing). However, not issuing and waiting for the completion of an IO barrier on fsync is broken behavior like you claim.

I am not sure how MacOS APFS behaves. I imagine that additional testing that takes into account the nuances in semantics would be able to clarify that. If it behaves like HFS+, it is broken.