zfs | amx | |
---|---|---|
17 | 18 | |
824 | 851 | |
0.0% | - | |
10.0 | 4.1 | |
about 4 years ago | 2 months ago | |
C | C | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
zfs
-
New System
https://openzfsonosx.org is a great starting place for general ZFS on Mac stuff.
-
How to handle an accidentally disconnected usb drive with a zfs pool on it?
I am imaging the following simple situation. I am using open ZFS on macOS (OpenZFS on OS X), and I have created a zpool with a single external USB drive. What command should I run when the drive is accidentally disconnected without properly exporting the pool?
-
Linux 6.2: The first mainstream Linux kernel for Apple M1 chips arrives
This is completely wrong. ZFS works very well via O3X [0]. I ran my home folder off of ZFS on various Mac Pros for ~11 years. ZFS works well in Linux and FreeBSD as well.
----
0: https://openzfsonosx.org/
-
Removable Time Machine Drive
This might be able to read the drive https://openzfsonosx.org/ But I don't know if Time Machine will jive with it.
-
OpenZFS on OS X
After dabbling with early stuff from the sadly scrapped Apple effort, I started my full time Mac ZFS journey in 2011 with the then Z-410, which then became ZEVO which was 10.8 only, and then staying on that (and in turn 10.8 doho) on a Mac Pro 3,1 for a solid 4 years or so. I was really sad when that effort didn't pan out and the company got acquired and then the whole deal was killed, but O3X revived the torch and I moved directly onto that. I've carried my pool forward continuously the whole way, with snapshots and everything, all data scrubbed and known good with no more data rot. It's a damn shame events (software patents) conspired to keep that from becoming a universal native fs, but it's still a wonderful thing and Lundman's amazing consistent efforts have been fantastic. The main site at https://openzfsonosx.org/ has decently active forums for something niche.
Without Apple on board or a much bigger effort ZFS on the Mac will probably always have some real limitations particularly with the GUI. For one of my desktop systems I've migrated to running my home and data folders as HFS or APFS formatted iSCSI targets, so it's still on ZFS underneath but appears native to macOS (and that also means that the effective death of towers isn't a limit on storage). But this remains an incredible project IMO.
Also for anyone who wants some limited GUI interaction capability, there is a small project called ZetaWatch [0] which will put some ZFS control into a menubar widget.
----
0: https://github.com/cbreak-black/ZetaWatch/
-
Question: JBOD and APFS. Is it still a bad idea?
OpenZFS is available for MacOS and it's about as rock solid as you get in a file system. As a bonus, if you someday need to migrate the dataset to a server, you'll be able to import the zfs pool pretty easily.
-
is there a Drivepool-type software for macOS?
Check out https://openzfsonosx.org/
-
[Q] What happened to the project ZFS on macOS?
Site ZFS on OSX seems down for the last few days...
-
Run FreeBSD 13.1 for ARM64 in QEMU on Apple Silicon Mac with HVF Acceleration
>such as ZFS
FWIW, ZFS has actually been available on macOS for a long time in various iterations (MacZFS, Z-410/ZEVO, and now OpenZFS [0]). I ran it for around 11 years on a number of systems with almost all of my data on it (home folder, applications, /opt for MacPorts etc) and it was a tank. Fun project and well worth checking out. I'll always regret the various factors that meant Apple didn't adopt it as their native FS.
----
0: https://openzfsonosx.org/
- The open source port of OpenZFS on OS X
amx
-
Optimize sgemm on RISC-V platform
I am talking about the matrix/vector coprocessor (AMX). You can find some reverse-engineered documentation here: https://github.com/corsix/amx
On M3 a singe matrix block can achieve ~ 1TFLOP on DGEMM, I assume it will be closer to 4TFLOPS for SGEMM. The Max variants have two such blocks. Didn't do precise benchmarking myself, but switching Python/R matrix libraries to use Apple's BLAS result in 5-6x perf improvement on matrix heavy code for me.
-
Intel AMX
It's really cool. I hope it becomes more common for training/inference/numerics capable accelerators to be included in consumer hardware.
Apple's AMX is really under-documented, while the instructions were reverse engineered, Virtually no benchmarks are available comparing current chip generations, models and variants.
https://github.com/corsix/amx
- Why do x86 processors take up so much energy when compared to ARM?
-
Bfloat16 support coming to Apple's Metal and PyTorch [video]
Visible in the unofficial documentation for AMX instructions too - M2 only bf16 functionality - https://github.com/corsix/amx/blob/main/matfp.md
-
LLaMA-7B in Pure C++ with full Apple Silicon support
Confusingly there are 2 mechanisms to do matrix operations on the new apple hardware - AMX (https://github.com/corsix/amx) - and the ANE (apple neural engine) - which is enabled by CoreML. This code does not run on the neural engine but the author has a branch for his whisper.cpp project which uses it here: https://github.com/ggerganov/whisper.cpp/pull/566 - so it may not be long before we see it applied here as well. All of this is to say that it actually could get significantly faster if some of this work was able to be handed to the ANE with CoreML.
-
Linux 6.2: The first mainstream Linux kernel for Apple M1 chips arrives
really? seems pretty well documented here: https://github.com/corsix/amx
-
AMX: The Secret Apple M1 Coprocessor
Article is almost two years old, and has a huge correction at the bottom. It's just a proprietary ISA extension, there's even a repo documenting what's been reverse engineered.
- corsix/amx: Apple AMX Instruction Set
-
Show HN: Port of OpenAI's Whisper model in C/C++
You are correct, in that those are the four
My understanding is that the AMX is more tightly wound with the CPU, ultimately being accessible via an instruction set (https://github.com/corsix/amx), and it is useful if you need to do matrix multiplications interleaved with other CPU tasks. A common example would be a VIO loop or something where you want that data in the CPU caches.
The GPU and Neural Engine are not that – they take some time to set up and initialize. They also can parallelize tasks to a much higher degree. The GPU is more generalizable, because you can write compute shaders to do anything in parallel, but it uses a lot of resources. I'll have to check out the PR to see how exactly the MPS shaders match up with the task at hand, because you could also consider writing Metal compute shaders by hand.
I know the least about the ANE, but it has specific hardware for running ML models, and you have to process the weights ahead of time to make sure they are in the right format. It can run ML models very efficiently and is the most battery friendly.
-
Ask HN: Are there any undocumented ISA extensions used in Linux systems?
If someone were to build a Linux system with proprietary ISA extensions, how would they do it given Linux is open source? Are there any examples of this being done? Would it be possible at all?
I got inspiration from this (https://github.com/corsix/amx) and I wondered if someone has done it before on a Linux-based system. I understand a userspace library could be created to access those instructions from userspace, but how would then they be implemented in the kernel? Through a proprietary kernel module built using a custom compiler? Or is that not needed at all and the library could just run on the processor taking advantage of the proprietary extensions?
What are some alternatives?
ZFSin - OpenZFS on Windows port
emacs-pure
zfs - OpenZFS on Linux and FreeBSD
whisper.cpp - Port of OpenAI's Whisper model in C/C++
hfs - HFS is a web file server to run on your computer. Share folders or even a single file thanks to the virtual file system.
sentencepiece - Unsupervised text tokenizer for Neural Network-based text generation.
archiso-zfs - Easily load ZFS kernel module on any Archiso.
whisper.cpp - Port of OpenAI's Whisper model in C/C++
MySQL - MySQL Server, the world's most popular open source database, and MySQL Cluster, a real-time, open source transactional database.
llama-mps - Experimental fork of Facebooks LLaMa model which runs it with GPU acceleration on Apple Silicon M1/M2
ZetaWatch - ZFS OSX Menu Bar widget
amx-rs - Rust wrapper for Apple Matrix Coprocessor (AMX) instructions