ollama VS jan

Download: Go to https://ollama.com/ and click on the download link for your operating system.

Most AI resume tools call out to OpenAI or Anthropic and charge you for every request. Persona supports Ollama — which means you can run the AI model locally on your own hardware, with zero API costs and zero data leaving your machine.

To solve these, we built a stack that prioritizes integrity over ease. The centerpiece is Ollama, running the mxbai-embed-large model locally. This is the engine that translates human thought into high-dimensional coordinates.

Ollama for model serving

Fully offline: Point it at Ollama or LM Studio. Zero cost, nothing leaves your network.

Recommended hardware: The RTX 3060 with 12 GB VRAM is the budget king here — all these models fit with room to spare for KV cache overhead, even Gemma 4:12B (which needs ~8.5–9 GB with overhead). An RTX 4060 Ti 16 GB gives you more headroom. On the Apple side, any M2 or M3 MacBook with 16 GB unified memory handles these models comfortably via Ollama's Metal backend.

This guide shows how to swap out every cloud API with a local Ollama server running qwen3-coder:30b. Same tools, same workflows, no data leaving your network.

Related: 35B MoE on 2× 1080 Ti · Ollama

The sample is a tiny console app running entirely against a local Ollama model — no cloud keys, and every HTTP call is traced so I can see exactly what goes over the wire (complete sample code). There's a single skill on disk:

irm https://ollama.com/install.ps1 | iex

Jan is the most polished open-source AI client available. Built with Tauri, it's lighter than Electron apps and has a genuinely clean, minimal design — the kind where you notice the absence of clutter rather than the presence of features. It runs local models through llama.cpp and MLX, has native MCP support, an extension system, and an OpenAI-compatible API server at localhost:1337 so you can point other tools at it.

Jan is a desktop application — Windows, macOS, Linux — that lets you download, manage, and chat with open-source LLMs without touching a terminal. Think of it as a ChatGPT replacement where the server runs on your machine. Licensed under Apache 2.0, with the full source on GitHub, it sits in the same category as LM Studio and Ollama, with a distinctly different philosophy: Jan wants to be the complete local AI platform, not just a runner or a chat UI.

Jan — offline-first desktop app

Jan takes a different approach, prioritizing user privacy and simplicity over advanced features with a 100% offline design that includes no telemetry and no cloud dependencies.

GitHub Repository: https://github.com/janhq/jan

Exactly: https://github.com/menloresearch/jan/issues/5474

Can't make it work with ollama endpoint

this looks to be the fix but they're not focusing on it: https://github.com/menloresearch/jan/issues/5474#issuecommen...

I will let you know tomorrow.

The front-end jan.ai now has a feature where it has an:

>Interface for uploading (or specifying) a folder, then running the prompt on all files in the folder

https://github.com/menloresearch/jan/issues/4909#event-18973...

Hopefully that will allow me to batch process checks/invoices to get them named appropriately, we'll see.

If you're on Mac you can download LM Studio and get the MLX version (qwen3-30b-a3b-instruct-2507-mlx). I am running it on 64 gb M1 and it takes about ~30 gb ram. I've been on the hunt for a local orchestrator model that interprets input with speech to text (STT) from WhisperX, then can decide what to do. I have only been running it for a day, but it may be overkill for my setup.

For simple tasks it can quickly respond and then understand to use MCP servers for tasks and other things, but offloading all the heavy lifting to claude code via sdk and cli, then bringing the results back in a summary or with clarifying questions as text to speech (TTS). I'm playing with Kyutai TTs b/c have great models that sound real and can do conversational streaming with VAD (though my mbp is too slow with it for now but see https://unmute.sh/ for demo).

I am looking for an orchestrator model that runs on 10-15 gb of ram and can do really good tool calling and model routing. I'm will likely move to something even smaller designed specifically for this, like Jan Nano and then spin up an intermediate model like Qwen if needed, or try a smaller Qwen. https://github.com/menloresearch/jan?tab=readme-ov-file

Ultimately, I want something that can see my screen and know what is going on and have full context and be live, so I was excited about Gemma 3N multi-modal, but its not really available yet fully with vision at least for MLX. https://deepmind.google/models/gemma/gemma-3n/

Next 6 months in this area is going to be pretty wild though.

https://github.com/menloresearch/jan