Dual-Edge-TPU-Adapter
neural-engine
Dual-Edge-TPU-Adapter | neural-engine | |
---|---|---|
10 | 20 | |
235 | 1,861 | |
- | - | |
1.2 | 5.1 | |
about 1 year ago | about 1 month ago | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Dual-Edge-TPU-Adapter
- Google Coral TPU
- Did someone succeed install Dual edge TPU?
- Ultra popular Linus Tech Tips abruptly drops their sponsor, Eufy Home Security Cameras, when it's revealed that Eufy has been secretly uploading images of the home owner, despite explicitly stating that the product only stores images locally.
-
5 PCIe devices on 10-12th gen mITX board?
M.2 Slot 4.0 x4 > This adapter that supports dual-bus E Key M.2 E Key dual Coral TPU
-
About to try the Google Coral Dual TPU (just half for now, waiting for the proper adapter to arrive)
The Coral Dual TPU uses both the lines of the E-key standard, but everything else just uses one. So basically every E-key slot can only use half of it. the adapter that I’m waiting is handmade from a guy just for this module. THIS GUY
-
M.2 E-Key to M-Key Adaptor with 2 PCIe Lanes for Dual Coral edge TPU?
while currently it is not in stock, this project provides various adapters for the dual coral (including a pcie adapter: https://github.com/magic-blue-smoke/Dual-Edge-TPU-Adapter)
-
Meet Mr. Big: 4TB SSD (2TB useable), 12.5Gbps, 288TB Disk storage (215TB useable), 32 cores
Get in on this: https://github.com/magic-blue-smoke/Dual-Edge-TPU-Adapter
-
Adding PCIe "Bifurcation" to an old Dell R720XD
Someone is trying to do this with these pcie-switch chips if you want to get on a waiting list for the adapter. Now I want to know if these pcie-switch chips can be chained...
-
ML at the edge - Google Coral ML PCIe Accelerator and Server 2022 Hyper-V DDA
Thanks so much for replying! Is the DDA working reliably? For some reason the Amazon links are not going through... I'm looking at picking up one of these too as my R530 doesn't support bifurcation: https://github.com/magic-blue-smoke/Dual-Edge-TPU-Adapter
neural-engine
-
Optimize sgemm on RISC-V platform
yep. they have a neural engine that is separate from the CPU and GPU that does really fast matmuls https://github.com/hollance/neural-engine. it's basically completely undocumented.
-
Apple is adding more and more neural engine cores to their products, is there any way to use them for local LLMs?
Looks like the ANE ("Apple Neural Engine") cores are powerful but not as flexible/programmable as the GPU cores. There is no sign that LLM inference is possible with them or ever will be unless Apple either opens up the closed ANE software framework for extensibility or they extend the ANE framework to support modern LLMs themselves. I would not hold my breath.
-
Anthropic’s $5B, 4-year plan to take on OpenAI
If Apple would wake up to what's happening with llama.cpp etc then I don't see such a big role for paying for remote access to big models via API
Currently a Macbook has a Neural Engine that is sitting idle 99% of the time and only suitable for running limited models (poorly documented, opaque rules about what ops can be accelerated, a black box compiler [1] and an apparent 3GB model size limit [2])
OTOH you can buy a Macbook with 64GB 'unified' memory and a Neural Engine today
If you squint a bit and look into the near future it's not so hard to imagine a future Mx chip with a more capable Neural Engine and yet more RAM, and able to run the largest GPT3 class models locally. (Ideally with better developer tools so other compilers can target the NE)
And then imagine it does that while leaving the CPU+GPU mostly free to run apps/games ... the whole experience of using a computer could change radically in that case.
I find it hard not to think this is coming within 5 years (although equally, I can imagine this is not on Apple's roadmap at all currently)
[1] https://github.com/hollance/neural-engine
- Everything we actually know about the Apple Neural Engine (ANE)
- What we know about the Apple Neural Engine
-
Everything we know about the Apple Neural Engine (ANE)
My question too. This semi-answer on the page seems to contradict itself (source: https://github.com/hollance/neural-engine/blob/master/docs/p... ):
"> Can I program the ANE directly?
Unfortunately not. You can only use the Neural Engine through Core ML at the moment.
There currently is no public framework for programming the ANE. There are several private, undocumented frameworks but obviously we cannot use them as Apple rejects apps that use private frameworks.
(Perhaps in the future Apple will provide a public version of AppleNeuralEngine.framework.)"
The last part links to this bunch of headers:
https://github.com/nst/iOS-Runtime-Headers/tree/master/Priva...
So might it be more accurate to say you can program it directly, but won't end up with something that can be distributed on the app store?
-
Apple VP Bob Borchers says Apple Silicon changed tech industry by pushing for energy efficiency
Read between their buzzwords. Apple's Neural Engine does nothing for training. It's purely for inference and it still requires the developers to go through their API. If a model uses a type layer Apple doesn't support, it's back to the CPU/GPU.
What are some alternatives?
edgetpu - Coral issue tracker (and legacy Edge TPU API source)
pyllms - Minimal Python library to connect to LLMs (OpenAI, Anthropic, AI21, Cohere, Aleph Alpha, HuggingfaceHub, Google PaLM2, with a built-in model performance benchmark.
Apple-Silicon-Guide - Apple Silicon Guide. Learn all about the A17 Pro, A16 Bionic, R1, M1-series, M2-series, and M3-series chips. Along with all the Devices, Operating Systems, Tools, Gaming, and Software that Apple Silicon powers.
ANECompat - A tool which checks compatibility of CoreML model with Apple Neural Engine
m2-module-jh-block-logger - Instruments HTML output to enable tools such as https://github.com/WeareJH/m2-dev-tools
pytorch-apple-silicon-benchmarks - Performance of PyTorch on Apple Silicon
pycoral - Python API for ML inferencing and transfer-learning on Coral devices
tensorexperiments - Boilerplate for GPU-Accelerated TensorFlow and PyTorch code on M1 Macbook
Virtualization-Documentation - Place to store our documentation, code samples, etc for public consumption.
more-ane-transformers - Run transformers (incl. LLMs) on the Apple Neural Engine.
project-birdfeeder
cnn-benchmarks - Benchmarks for popular CNN models