iOS-Runtime-Headers
tinygrad
| iOS-Runtime-Headers | tinygrad | |
|---|---|---|
| 2 | 58 | |
| 7,923 | 17,800 | |
| - | - | |
| 10.0 | 9.7 | |
| almost 2 years ago | 10 months ago | |
| Objective-C | Python | |
| - | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
iOS-Runtime-Headers
-
Android Devices with Backdoored Firmware Found in US Schools
Sure, but private methods are another vector - tracking and bypassing the IDFA and potentially acting as official Apple Apps to use/abuse things like Carrier/SIM info[0], updating the wallpaper for the user[1], accessing call history[2], etc.
0: https://github.com/nst/iOS-Runtime-Headers/blob/fbb634c78269...
1: https://github.com/nst/iOS-Runtime-Headers/issues/32
2: https://github.com/nst/iOS-Runtime-Headers/tree/fbb634c78269...
-
Everything we know about the Apple Neural Engine (ANE)
My question too. This semi-answer on the page seems to contradict itself (source: https://github.com/hollance/neural-engine/blob/master/docs/p... ):
"> Can I program the ANE directly?
Unfortunately not. You can only use the Neural Engine through Core ML at the moment.
There currently is no public framework for programming the ANE. There are several private, undocumented frameworks but obviously we cannot use them as Apple rejects apps that use private frameworks.
(Perhaps in the future Apple will provide a public version of AppleNeuralEngine.framework.)"
The last part links to this bunch of headers:
https://github.com/nst/iOS-Runtime-Headers/tree/master/Priva...
So might it be more accurate to say you can program it directly, but won't end up with something that can be distributed on the app store?
tinygrad
- tinygrad: extreme simplicity, easiest framework to add new accelerators to
-
GGML – AI at the Edge
Might be a silly question but is GGML a similar/competing library to George Hotz's tinygrad [0]?
[0] https://github.com/geohot/tinygrad
-
Render neural network into CUDA/HIP code
at first glance i thought may its like tinygrad. but looks has many ops than that tiny grad but most maps to underlying hardware provided ops?
i wonder how well tinygrad's apporach will work out, ops fusion sounds easy, just a walk a graph, pattern match it and lower to hardware provided ops?
Anyway if anyone wants to understand the philosophy behind tinygrad, this file is great start https://github.com/geohot/tinygrad/blob/master/docs/abstract...
-
llama.cpp now officially supports GPU acceleration.
There are currently at least 3 ways to run llama on m1 with GPU acceleration. - mlc-llm (pre-built, only 1 model has been ported) - tinygrad (very memory efficient, not that easy to integrate into other projects) - llama-mps (original llama codebase + llama adapter support)
- George Hotz building an AMD competitor to Nvidia.
-
George Hotz ROCm adventures
Hopefully we will see now full support with AMD hardware on https://github.com/geohot/tinygrad. You can read more about it on https://tinygrad.org/
-
The Coming of Local LLMs
tinygrad
https://github.com/geohot/tinygrad/tree/master/accel/ane
But I have not tested it on Linux since Asahi has not yet added support.
llama.cpp runs at 18ms per token (7B) and 200ms per token (65B) without quantization.
- Everything we know about Apple's Neural Engine
- Everything we know about the Apple Neural Engine (ANE)
- How 'Open' Is OpenAI, Really?
What are some alternatives?
neural-engine - Everything we actually know about the Apple Neural Engine (ANE)
Pytorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration
ane - Reverse engineered Linux driver for the Apple Neural Engine (ANE).
llama.cpp - LLM inference in C/C++
m1n1 - A bootloader and experimentation playground for Apple Silicon
openpilot - openpilot is an open source driver assistance system. openpilot performs the functions of Automated Lane Centering and Adaptive Cruise Control for 250+ supported car makes and models.
ml-ane-transformers - Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE)
llama - Inference code for Llama models
whisper.cpp - Port of OpenAI's Whisper model in C/C++
tensorflow_macos - TensorFlow for macOS 11.0+ accelerated using Apple's ML Compute framework.
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.