ollama VS openrouter-runner

Download: Go to https://ollama.com/ and click on the download link for your operating system.

Most AI resume tools call out to OpenAI or Anthropic and charge you for every request. Persona supports Ollama — which means you can run the AI model locally on your own hardware, with zero API costs and zero data leaving your machine.

To solve these, we built a stack that prioritizes integrity over ease. The centerpiece is Ollama, running the mxbai-embed-large model locally. This is the engine that translates human thought into high-dimensional coordinates.

Ollama for model serving

Fully offline: Point it at Ollama or LM Studio. Zero cost, nothing leaves your network.

Recommended hardware: The RTX 3060 with 12 GB VRAM is the budget king here — all these models fit with room to spare for KV cache overhead, even Gemma 4:12B (which needs ~8.5–9 GB with overhead). An RTX 4060 Ti 16 GB gives you more headroom. On the Apple side, any M2 or M3 MacBook with 16 GB unified memory handles these models comfortably via Ollama's Metal backend.

This guide shows how to swap out every cloud API with a local Ollama server running qwen3-coder:30b. Same tools, same workflows, no data leaving your network.

Related: 35B MoE on 2× 1080 Ti · Ollama

The sample is a tiny console app running entirely against a local Ollama model — no cloud keys, and every HTTP call is traced so I can see exactly what goes over the wire (complete sample code). There's a single skill on disk:

irm https://ollama.com/install.ps1 | iex

from langchain_openai import ChatOpenAI from langchain_community.tools import DuckDuckGoSearchRun from langchain.tools import Tool from langchain.memory import ConversationBufferMemory from langchain.agents.agent_types import AgentType from langchain.agents import initialize_agent from dotenv import load_dotenv import os load_dotenv() def webSearchAgent(question): llm = ChatOpenAI( model = "deepseek/deepseek-chat-v3-0324:free", base_url = "https://openrouter.ai/api/v1", api_key = os.getenv("OPENROUTER_API_KEY"), ) search = DuckDuckGoSearchRun() tool = [ Tool ( name = "search", func = search.run, description = "When you want real time data use this", ) ] memory = ConversationBufferMemory(memory_key = "chat_history") agent = initialize_agent( llm = llm, tools = tool, agent = AgentType.CONVERSATIONAL_REACT_DESCRIPTION, memory = memory, verbose = True, handle_parsing_errors = True ) result = agent.run(question) print(f"Agent:\n\n {result}") webSearchAgent("Give me some latest news about Ai")

OpenRouter API Key → Create an account at OpenRouter and grab your key

OpenRouter (multi-model API wrapper) https://openrouter.ai

{ "Providers": [ { "name": "openrouter", "api_base_url": "https://openrouter.ai/api/v1/chat/completions", "api_key": "sk-xxx", "models": [ "anthropic/claude-sonnet-4" ], "transformer": { "use": ["openrouter"] } } ], "Router": { "default": "openrouter,anthropic/claude-sonnet-4" }, "LOG": true, "HOST": "127.0.0.1", + "PORT": 23456 }

OpenRouter - We love OpenRouter because it allows you to easily try out new models and load balance between models. We actually got an open source contribution for this one recently, so we should be supporting it ❤️

Link: Promptfoo Open Source Tool for Evaluation and Red Teaming Link: OpenRouterFor Moonshot Kimi K2 APIs Link: Kimi K2 Model Model Page

Then I discovered OpenRouter: "a unified API platform that provides access to a wide range of large language models with intelligent routing and automatic fallbacks." You can use whatever model you want with the same API key. But the feature I really needed was its provisionary API key system, which allowed me to generate one master key and programmatically:

What’s different is all the open weight models like Kimi-k2 or Qwen-3 Coder that are as good and, depending on the task, better than Anthropic’s Sonnet model for 80% less via openrouter [1] and other similar services.

You can use these models through Claude Code; I do it everyday.

Some developers are running smaller versions of these LLMs on their own hardware, paying no one.

So I don’t think Anthropic and the other companies can dramatically increase their prices without losing the customers that helped them go from $0 to $4 billion in revenue in 3 years.

Users can easily move between different AI platforms with no lock-in, which makes it harder to increase prices and proceed to enshitify their platforms.

[1]: https://openrouter.ai/