M2 Ultra can run 128 streams of Llama 2 7B in parallel

• llama.cpp

  772 56,891 10.0 C++

  LLM inference in C/C++

• whisper.coreml

  1 6 8.5 Python

  Robust Speech Recognition via Large-Scale Weak Supervision

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• more-ane-transformers

  4 35 7.0 Python

  Run transformers (incl. LLMs) on the Apple Neural Engine.

• ollama

  196 62,615 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

They are very much uncensored, give it a try yourself: `ollama run llama2-uncensored` [0]

It will be happy to curse, talk about religions, help you cook illicit substances or do all sorts of other stuff.

[0] https://ollama.ai/

Suggest a related project