Apple is adding more and more neural engine cores to their products, is there any way to use them for local LLMs?

• neural-engine

  20 1,866 5.1

  Everything we actually know about the Apple Neural Engine (ANE)

Looks like the ANE ("Apple Neural Engine") cores are powerful but not as flexible/programmable as the GPU cores. There is no sign that LLM inference is possible with them or ever will be unless Apple either opens up the closed ANE software framework for extensibility or they extend the ANE framework to support modern LLMs themselves. I would not hold my breath.

• ml-ane-transformers

  22 2,461 0.0 Python

  Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE)

  

Article: Deploying Transformers on the Apple Neural Engine Code: Apple Neural Engine (ANE) Transformers

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project