How to Setup a Local Coding Agent on macOS

1. llama.cpp

   1 1,050 120,506 10.0 C++

   LLM inference in C/C++

   

   > The benchmark prompt was:

   > Write a compact Python function that parses a unified diff and returns the changed file paths. Then explain two edge cases.

   > Each benchmark generated about 128 tokens.

   Generating 128 tokens is probably not enough for good benchmark results. MTP speedup depends on how often the predicted tokens are accepted. In my experience, the very early output has a higher acceptance rate, so short testing can give false positive speedups.

   Also llama.cpp includes a tool specifically for benchmarking:

   https://github.com/ggml-org/llama.cpp/blob/master/tools/llam...

2. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. pi-sandbox

   2 1 15 7.0 Shell

   A Docker Sandbox for running the Pi agent locally with a model served by oMLX. (by Dotnaught)

   

4. omlx

   3 22 17,762 9.9 Python

   LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

   

   M4 24GB here. You'll be fine, if you're anything like me minor latency is acceptable to obtain (a) privacy (b) reliability (c) CI/CD/guardrails (d) network independence (e) future-proofing vs. AIaaS. https://omlx.ai/ gives you intelligent local hardware based model download recommendations.

5. ds4

   4 32 53 - C

   DeepSeek 4 Flash local inference engine for Metal

   

   I've been quite impressed with DeepSeek v4 Flash running via antirez's ds4[0].

   It feels like a GPT-4 class model in terms of "stored knowledge" but is better at long-horizon tool calling than any of the GPT-4 class models.

   Running on a 128GB MBP M4 Max, I'm getting ~24 t/s on generation and ~200 t/s on prefill. I was expecting it to feel slow, and it certainly does when e.g. generating code, but it's surprisingly useful as a "machine orchestrator" for simple tasks.

   For non-agentic usecases, it's a surprisingly decent model to converse with, and has the benefit of being entirely self-contained/private.

   [0]https://github.com/antirez/ds4

6. ds4

   5 1 4 6.3 C

   DeepSeek 4 Flash local inference engine for Metal (by ljubomirj)

   

   For high Ram (unified), and relatively middling to lowish Tflops and bandwidth GB/s, usually MoEs are most hopeful. The current top-1 in the (iq, tok/s, @ context depth) ranks for me (M2 Max, 96gb) is DeepSeek-V4-Flash REAP25 <65gb gguf + ds4-server + pi agent. Not better than cloud API ofc, but useful enough to endure if I need to. E.g on a non-Internet 4h flight the battery (local llm draws 60w) held long enough. REAP supporting ds4 branch here

   https://github.com/ljubomirj/ds4/tree/reap-compact-support

   DS4F dropping to unusable <10 tok/s only at 784K context (!!) makes a big difference.

Suggest a related project