Is llama.cpp any good on ARM (e.g. Ampere Altra) or only on x86-64?

• ml-ane-transformers

  22 2,461 0.0 Python

  Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE)

  

I'm also looking into converting these models from PyTorch to CoreML format, and seeing how well they run when given access to the GPU and Neural Engine. There's even an optimized library Apple has specifically for this type of model.

• llama.cpp

  775 57,463 10.0 C++

  LLM inference in C/C++

Perf on RK3588 on FriendlyElec NanoPi R6s: https://github.com/ggerganov/llama.cpp/issues/722

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project