Textgen Alternatives

textgen reviews and mentions

• Is there an IDE that can use the local open-source model?
  5 projects | news.ycombinator.com | 26 Oct 2025

  I've been looking for this too. It seems to me as though all the ide's are trying to sell the llms as a service or trying to lock you in by downloading llms through their ide. I have been downloading llm's from huggingface as gguf files and would like to use those downloads (and running them through https://github.com/oobabooga/text-generation-webui). It is possible to run those llms as a local api using something like llama-cpp-python and would prefer to use something like that method. Zed (https://zed.dev/), which is now available on windows might be able to do it, but i'd rather use something (foss) that doesn't have a pricing model (the development focus will always be upon those who pay). tbh i'm getting a bit sick of changing ide's, as their support changes, and really would prefer not to use (microsoft) visual studio code which seems to be cornering the market. Starting to think i'm going to try to learn emacs, with https://github.com/karthink/gptel looking as if it would meet my needs.

• When you're asking AI chatbots for answers, they're data-mining you
  2 projects | news.ycombinator.com | 18 Aug 2025

  There are also things like Oobabooga's text-generation-webui[0] which can present a similar interface to ChatGPT for local models.

  I've had great success in running Qwen3-8B-GGUF[1] on my RTX 2070 SUPER (8GB VRAM) using Oobabooga (everyone just calls it via the author's name, it's much catchier) so this is definitely doable on consumer hardware. Specifically, I run the Q4_K_M model as Oobabooga loads all of its layers into the GPU by default, making it nice and snappy. (Testing has shown that I can actually load up to the Q6_K model before some layers have to be loaded into the CPU, but I have to manually specify that all those layers should be loaded into the GPU, as opposed to leaving it auto-determined.)

  It does obviously hallucinate more often than ChatGPT does, so care should be taken. That said, it's really nice to have something local.

  There's a subreddit for running text gen models locally that people might be interested in: https:// www.reddit.com/r/LocalLLaMA

  [0] https://github.com/oobabooga/text-generation-webui

  [1] https://huggingface.co/Qwen/Qwen3-8B-GGUF

• How to Install NVIDIA AceReason-Nemotron-14B Locally?
  1 project | dev.to | 3 Jun 2025

  git clone https://github.com/oobabooga/text-generation-webui cd text-generation-webui

• 1,156 Questions Censored by DeepSeek
  1 project | news.ycombinator.com | 28 Jan 2025

  total time = 392339.02 ms / 2221 tokens

  And my exact command was:

  llama-server --model DeepSeek-R1-UD-Q2_K_XL-00001-of-00005.gguf --temp 0.6 -c 9000 --min-p 0.1 --top-k 0 --top-p 1 --timeout 3600 --slot-save-path ~/llama_kv_path --port 8117 -ctk q8_0

  (IIRC slot save path argument does absolutely nothing unless and is superfluous, but I have been pasting a similar command around and been too lazy to remove it). -ctk q8_0 reduces memory use a bit for context.

  I think my 256gb is right at the limit of spilling a bit into swap, so I'm pushing the limits :)

  To explain to anyone not aware of llama-server: it exposes (a somewhat) OpenAI-compatible API and then you can use it with any software that speaks that. llama-server itself also has a UI, but I haven't used it.

  I had some SSH tunnels set up to use the server interface with https://github.com/oobabooga/text-generation-webui where I hacked an "OpenAI" client to it (that UI doesn't have it natively). The only reason I use the oobabooga UI is out of habit so I don't recommend this setup to others.

• DeepSeek-R1 with Dynamic 1.58-bit Quantization
  3 projects | news.ycombinator.com | 28 Jan 2025

  Can't this kind of repetition be dealt with at the decoder level, like for any models? (see DRY decoder for instance: https://github.com/oobabooga/text-generation-webui/pull/5677)

• I Run LLMs Locally
  5 projects | news.ycombinator.com | 29 Dec 2024

  Still nothing better than oobabooga (https://github.com/oobabooga/text-generation-webui) in terms of maximalism/"Pro"/"Prosumer" LLM UI/UX ALA Blender, Photoshop, Final Cut Pro, etc.

  Embarrassing and any VCs reading this can contact me to talk about how to fix that. lm-studio is today the closest competition (but not close enough) and Adobe or Microsoft could do it if they fired their current folks which prevent this from happening.

  If you're not using Oobabooga, you're likely not playing with the settings on models, and if you're not playing with your models settings, you're hardly even scratching the surface on its total capabilities.

• Yi-Coder: A Small but Mighty LLM for Code
  5 projects | news.ycombinator.com | 5 Sep 2024

  I understand your situation. It sounds super simple to me now but I remember having to spend at least a week trying to get the concepts and figuring out what prerequisite knowledge I would need between a continium of just using chatgpt and learning relevant vector math etc. It is much closer to the chatgpt side fortunately. I don't like ollama per se (because i can't reuse its models due to it compressing them in its own format) but it's still a very good place to start. Any interface that lets you download models as gguf from huggingface will do just fine. Don't be turned off by the roleplaying/waifu sounding frontend names. They are all fine. This is what I mostly prefer: https://github.com/oobabooga/text-generation-webui

• XTC: An LLM sampler that boosts creativity, breaks writing clichés
  1 project | news.ycombinator.com | 18 Aug 2024
• Codestral Mamba
  15 projects | news.ycombinator.com | 16 Jul 2024

  Why do people recommend this instead of the much better oobabooga text-gen-webui?

  https://github.com/oobabooga/text-generation-webui

  It's like you hate settings, features, and access to many backends!

• Why I made TabbyAPI
  4 projects | dev.to | 12 Jul 2024

  The issue is running the model. Exl2 is part of the ExllamaV2 library, but to run a model, a user needs an API server. The only option out there was using text-generation-webui (TGW), a program that bundled every loader out there into a Gradio webui. Gradio is a common “building-block” UI framework for python development and is often used for AI applications. This setup was good for a while, until it wasn’t.

• A note from our sponsor - SaaSHub
  www.saashub.com | 11 Jun 2026

  SaaSHub helps you find the best software and product alternatives Learn more →