ollama VS khoj

Download: Go to https://ollama.com/ and click on the download link for your operating system.

Most AI resume tools call out to OpenAI or Anthropic and charge you for every request. Persona supports Ollama — which means you can run the AI model locally on your own hardware, with zero API costs and zero data leaving your machine.

To solve these, we built a stack that prioritizes integrity over ease. The centerpiece is Ollama, running the mxbai-embed-large model locally. This is the engine that translates human thought into high-dimensional coordinates.

Ollama for model serving

Fully offline: Point it at Ollama or LM Studio. Zero cost, nothing leaves your network.

Recommended hardware: The RTX 3060 with 12 GB VRAM is the budget king here — all these models fit with room to spare for KV cache overhead, even Gemma 4:12B (which needs ~8.5–9 GB with overhead). An RTX 4060 Ti 16 GB gives you more headroom. On the Apple side, any M2 or M3 MacBook with 16 GB unified memory handles these models comfortably via Ollama's Metal backend.

This guide shows how to swap out every cloud API with a local Ollama server running qwen3-coder:30b. Same tools, same workflows, no data leaving your network.

Related: 35B MoE on 2× 1080 Ti · Ollama

The sample is a tiny console app running entirely against a local Ollama model — no cloud keys, and every HTTP call is traced so I can see exactly what goes over the wire (complete sample code). There's a single skill on disk:

irm https://ollama.com/install.ps1 | iex

A concrete example from our scan: Khoj, an open-source AI assistant, exposes a function called ai_update_memories that lets the LLM delete and replace user memories. It calls session.delete() followed by session.add() with no confirmation, no rate limit, and no validation on the content. A single adversarial prompt could wipe and replace a user's entire memory store.

GitHub https://github.com/khoj-ai/khoj GitHub Star 12.4k GitHub Fork 627 GitHub Issue 64 GitHub Pull Request 3 GitHub Contributor 35 Open Source License AGPL-3.0 Official Website https://khoj.dev/ Documentation https://docs.khoj.dev/

There are already several RAG chat open source solutions available. Two that immediately come to mind are:

Danswer

https://github.com/danswer-ai/danswer

Khoj

https://github.com/khoj-ai/khoj

I'm a fan of Khoj. Been using it for months. https://github.com/khoj-ai/khoj

https://github.com/khoj-ai/khoj

This is the easiest I found, on here too.

Thanks for the feedback. Does your machine have a GPU? 32GB CPU RAM should be enough but GPU speeds up response time.

We have fixes for the seg fault[1] and improvement to the query speed[2] that should be released by end of day today[3].

Update khoj to version 0.10.1 with pip install --upgrade khoj-assistant to see if that improves your experience.

The number of documents/pages/entries doesn't scale memory utilization as quickly and doesn't affect the search, chat response time as much

[1]: The seg fault would occur when folks sent multiple chat queries at the same time. A lock and some UX improvements fixed that

[2]: The query time improvements are done by increasing batch size, to trade-off increased memory utilization for more speed

[3]: The relevant pull request for reference: https://github.com/khoj-ai/khoj/pull/393

We recently integrated Llama 2 into Khoj. I wanted to share a short real-world evaluation of using Llama 2 for the chat with docs use-cases and hear which models have worked best for you all. The standard benchmarks (ARC, HellaSwag, MMLU etc.) are not tuned for evaluating this