ollama VS mlx

Download: Go to https://ollama.com/ and click on the download link for your operating system.

Most AI resume tools call out to OpenAI or Anthropic and charge you for every request. Persona supports Ollama — which means you can run the AI model locally on your own hardware, with zero API costs and zero data leaving your machine.

To solve these, we built a stack that prioritizes integrity over ease. The centerpiece is Ollama, running the mxbai-embed-large model locally. This is the engine that translates human thought into high-dimensional coordinates.

Ollama for model serving

Fully offline: Point it at Ollama or LM Studio. Zero cost, nothing leaves your network.

Recommended hardware: The RTX 3060 with 12 GB VRAM is the budget king here — all these models fit with room to spare for KV cache overhead, even Gemma 4:12B (which needs ~8.5–9 GB with overhead). An RTX 4060 Ti 16 GB gives you more headroom. On the Apple side, any M2 or M3 MacBook with 16 GB unified memory handles these models comfortably via Ollama's Metal backend.

This guide shows how to swap out every cloud API with a local Ollama server running qwen3-coder:30b. Same tools, same workflows, no data leaving your network.

Related: 35B MoE on 2× 1080 Ti · Ollama

The sample is a tiny console app running entirely against a local Ollama model — no cloud keys, and every HTTP call is traced so I can see exactly what goes over the wire (complete sample code). There's a single skill on disk:

irm https://ollama.com/install.ps1 | iex

I thought this seemed significant at the time: https://github.com/ml-explore/mlx/pull/1983

Compilation is cleaner on dense. llama.cpp, MLX, and vLLM all support both, but the dense path has had more attention. Fewer corner cases in expert routing, GQA, and KV layout interactions. If you've ever had a custom kernel mis-handle expert dispatch, you know.

For AI, you don't have to use a cloud model. Ollama and Apple MLX let you run models locally against the same folder. Useful if you have notes you'd rather not send to an external API. DS4 is worth looking at specifically. The latest models support up to 200k token context windows, so you can feed in most of your notes folder in a single pass.

Further reading: Common AI Model Formats (HuggingFace Blog) · What is GGUF? Complete Guide · Safetensors Security Audit · MLX GitHub · Ollama: Importing Models

MLX Framework (Apple) — Apple's native ML framework for Apple Silicon

The --oss flag changes everything about how Codex CLI operates. Instead of calling the OpenAI API, it connects to a local model runtime. Ollama, LM Studio, and MLX are all supported through the OpenAI-compatible API interface.

Framework: MLX v0.28.0

(https://github.com/ml-explore/mlx).

GitHub: https://github.com/skryl/mlx-ruby

MLX-Ruby is a native C++ extension that wraps the upstream MLX runtime, giving Ruby full access to the array framework, neural network layers, optimizers, and Metal GPU acceleration on Apple silicon.

What’s included:

MLX is Apple's machine learning framework, built specifically for Apple Silicon chips (M1, M2, M3, M4, and their Pro/Max/Ultra variants).

Sure, I directly and explicitly talked about Apple's version of tensor cores in the GPU. But the ANE is by every definition a neural accelerator. Yes, I'm aware of Apple's weird branding for their tensor cores.

"In fact MLX does not even support ANE yet"

I didn't say otherwise. The ANE is a fantastic unit for small, power-efficient models, like extracting text from images, doing depth modelling, etc. It's not made for LLMs, or the other sorts of experimental stuff MLX is intended for. Though note that MLX's author's reason for not supporting the ANE is that it has a "closed-source" API (https://github.com/ml-explore/mlx/issues/18#issuecomment-184...), making it unsuitable for an open-source project. But anyways, the ANE is fantastically fast at what it does, while sipping juice.

In any case, the code change shown should have zero impact on the running of MLX on an iPhone 16 Pro. MLX tries to really leverage platform optimizations so maybe another bifucation is making the wrong choice.