Llamafile 0.7 Brings AVX-512 Support: 10x Faster Prompt Eval Times for AMD Zen 4

• llamafile

  39 16,345 9.7 C++

  Distribute and run LLMs with a single file.

  

I just had a look at their Github, and true enough:

“External weights are particularly useful for Windows users because they enable you to work around Windows' 4GB executable file size limit.

For Windows users, here's an example for the Mistral LLM:

curl -L -o llamafile.exe https://github.com/Mozilla-Ocho/llamafile/releases/download/...

curl -L -o mistral.gguf https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGU...

./llamafile.exe -m mistral.gguf -ngl 9999

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• highway

  68 3,741 9.8 C++

  Performance-portable, length-agnostic SIMD with runtime dispatch

  

The bf16 dot instruction replaces 6 instructions: https://github.com/google/highway/blob/master/hwy/ops/x86_12...

Suggest a related project