zfs VS middleware

https://github.com/openzfs/zfs/issues/15526#issuecomment-181...

> zpool get all tank | grep bclone

> kc3000 bcloneused 442M

> kc3000 bclonesaved 1.42G

> kc3000 bcloneratio 4.30x

> My understanding is this: If the result is 0 for both bcloneused and bclonesaved then it's safe to say that you don't have silent corruption.

It's a very rare race condition, odds are very low that you were impacted. If you were, you would have noticed (heavy builds with files being moved around where suddenly files are zero).

[0] https://bugs.gentoo.org/917224

[1] https://github.com/openzfs/zfs/issues/15526 (referenced in the article)

A couple years ago, I had an idea for convincing a filesystem to go faster using 2 compression steps instead of one. I couldn't see why it wouldn't work, and I also couldn't convince myself it should.

It seems to have worked out. [1]

[1] - https://github.com/openzfs/zfs/commit/f375b23c026aec00cc9527...

https://github.com/openzfs/zfs/issues/7631

This is a long-standing issue with zvols which affects overall system stability, and has no real solution as of yet.

More details on this discussion: https://github.com/openzfs/zfs/issues/6824#issuecomment-1817... . Basically he is too busy to continue developing the encryption features but is able to review the related works.

There is also discussion about using ZFS on LUKS in the same thread: https://github.com/openzfs/zfs/issues/6824#issuecomment-1819...

ZFS on top of LUKS seems to have it's own issues though. :(

https://github.com/openzfs/zfs/issues/15533

https://github.com/openzfs/zfs/pull/15529#pullrequestreview-...

Honestly, ZFS is the best thing on the (Free)BSDs only... On Linux it doesn't even use the page cache, and you conflict severely with L2ARC. I know there's a variety of people who don't care, but still for real users it's not an actual option.

A more "correct" fix has been posted https://github.com/openzfs/zfs/pull/15615

Current Master: https://github.com/openzfs/zfs/blob/acb33ee1c169bf1c1f687db1...

When I look up the problem, I could only see the issue being discussed, and probably leading to that commit, and then it was not in my current release (2.1) despite several years later. I’m wondering if ZFS still holds that high standard for reliability.

For storage options, most people either purchase a NAS (network attached storage) or re-purpose an older computer using either TrueNAS or unraid. If you're looking to just purchase one, the most popular brand is synology, but their models can be a bit pricey.

The second is storage. If you need any of the storage sharing, deploy NAS OS as the VM in proxmox, like Starwinds SAN and NAS (https://www.starwindsoftware.com/san-and-nas) or OMV (https://www.openmediavault.org/), or TrueNAS (https://www.truenas.com/). As you mentioned, you need to cross-flash the perc into IT mode and pass through the controller into VM, but you need a separate from the controller drive for proxmox to be able to PCI-E passthrough the card into VM. Then, configure software RAID and reshare the storage to the proxmox via NFS/iSCSI (that will improve your skills in storage stack and storage protocols).

A quick mention of a few open source projects not in the list but we do use for both our MSP and enterprise clients are TrueNAS & Xen Orchestra / XCP-NG

The middleware source code is available on GitHub: https://github.com/truenas/middleware

If you want something more like NAS or shared folder, you should check out TrueNas. You can build your own system from a PC and a few hard drives, or go for a bit more professional solution and buy a NAS hardware and disks from them.

I would also like some kind of start page, and monitoring tools. Homepage looks like it may be a good start along with Scrutiny and possibly Cockpit. I do want this as a NAS too and was tempted to put TrueNAS on so I could manage ZFS and shares but suspect I can do that another way without the overhead. (either terminal commands, or poss through Cockpit)

If you are comfortable with learning something complex I would get two Truenas systems set up one locally (at the studio) and one at your friend's home so that they sync to each other (creating more redundancy) and if you don't know networking off the back of your hand I would say set up two Unifi DMP and do a site to site connection so it's more secure.

The solution is in the link I posted (use systemd-nspawn to partition a userspace jail to run Docker in, easier than you think); as for "official" statement from Ix Systems it was first mentioned here but this link is to the definative commit where it is removed.

If you asked me 3 years ago, what distribution I would recommend as NAS - I would no-doubt answer: TrueNAS, of course!

As nas os running inside of the cluster you can look into something like TrueNAS (https://www.truenas.com/), OMV (https://www.openmediavault.org/) or Starwinds san and nas, they have also guide for hyper-v setup (https://www.starwindsoftware.com/resource-library/starwind-san-and-nas/)