I made an app that runs Mistral 7B 0.2 LLM locally on iPhone Pros

1. Cgml

   1 31 56 8.6 C++

   GPU-targeted vendor-agnostic AI library for Windows, and Mistral model implementation.

   

   Is that explanation better? https://github.com/Const-me/Cgml/blob/master/Mistral/Mistral...

   Same Mistral Instruct 0.2 model, different implementation.

2. Judoscale

   judoscale.com featured

   Save 47% on cloud hosting with autoscaling that just works. Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.

   

3. llama.cpp

   2 883 78,821 10.0 C++

   LLM inference in C/C++

   

   We should be happy that compute is once again improving and machines are getting outdated rapidly. Which is better - a world where your laptop is competitive for 5+ years but everything stays the same? Or one where entire new realms of advancement open up every 18 months?

   It’s a no contest option 2 for me.

   Just use llama.cpp with any of the available UIs. It will be usable with 4 but quantization on CPU. You can use any of the “Q4_M” “GGUF” models that TheBloke puts out on Huggingface.

   https://github.com/ggerganov/llama.cpp

   UI projects in description.

   https://huggingface.co/TheBloke

   A closed source option is LMStudio.

   https://lmstudio.ai/

4. exporters

   3 3 656 2.7 Python

   Export Hugging Face models to Core ML and TensorFlow Lite

   

5. enchanted

   4 8 5,218 6.3 Swift

   Enchanted is iOS and macOS app for chatting with private self hosted language models such as Llama2, Mistral or Vicuna using Ollama.

   

6. llama_index

   5 78 41,098 9.9 Python

   LlamaIndex is the leading framework for building LLM-powered agents over your data.

   

   Mistral Instruct does use a system prompt.

   You can see the raw format here: https://www.promptingguide.ai/models/mistral-7b#chat-templat... and you can see how LllamaIndex uses it here (as an example): https://github.com/run-llama/llama_index/blob/1d861a9440cdc9...

7. swift-transformers

   6 2 923 8.3 Swift

   Swift Package to implement a transformers-like API in Swift

   

8. mlx

   7 28 20,281 9.8 C++

   MLX: An array framework for Apple silicon

   

9. InfluxDB

   influxdata.com featured

   InfluxDB high-performance time series database. Collect, organize, and act on massive volumes of high-resolution data to power real-time intelligent systems.

   

10. ollama

    8 463 138,252 9.9 Go

    Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1 and other large language models.

    

    ollamma https://ollama.ai/ is popular choice for running local llm models and should work fine on intel. It's just wrapping docker so shouldn't require m2/m3.

Suggest a related project