Jupyter Notebook Bigdata Projects
CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.Project mention: The Wiretrustee SATA Pi Board Is a True SATA NAS | news.ycombinator.com | 2021-06-10
I keep hoping some day the drives will have their own networking built in. Kioxia, a Toshiba spin off, announced a network-attached NVMe-oF drive last September, and I seem to recall one of the major drive players had similar intents a bit back... ah yes, the Seagate Kinetic drives with dual 1Gbit & an object storage OS built in to the drive. These days Seagate seems to be pushing a software platform CORTX, which I hope some day perhaps has hardware products too (but right now seems to be for classic linux-based network appliances)
Ideally we start using 5 or 10Gbit ethernet for these cases. We could continue to treat these drives like they are direct attached, even though they are network attached, and either have one computer running RAID, or have Ceph and a bunch of computers running it's distributed system to tap the drives.
Ideally though, we need new clustered file-systems, where any computer can read the drives. That is, I'd guess, a long way off. Legacy devices (home media players) would need to go through some kind of legacy gateway.
Are you hiring? Post a new remote job listing for free.