GPTQ-for-LLaMa
langflow
GPTQ-for-LLaMa | langflow | |
---|---|---|
19 | 28 | |
129 | 17,467 | |
- | 12.6% | |
7.7 | 10.0 | |
11 months ago | 2 days ago | |
Python | JavaScript | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
GPTQ-for-LLaMa
-
I have tried various different methods to install, and none work. Can you spoon-feed me how?
git clone https://github.com/oobabooga/GPTQ-for-LLaMa
-
Query output random text
If you're using the model directly from ehartford, that one hasn't been quantized. Try using the GPTQ quantized version here, and use this fork of GPTQ-for-LLaMa. Load in 4-bit with --wbits 4
-
Help needed with installing quant_cuda for the WebUI
This worked for me on Ubuntu. If you want to use the CUDA branch instead of triton, do the same steps except clone this GPTQ-for-LLaMa fork and run python setup_cuda.py install
-
AutoGPTQ vs GPTQ-for-llama?
If you don't have triton and you use AutoGPTQ you're gonna notice a huge slow down compared to the old GPTQ-for-LLaMA cuda branch. For me AutoGPTQ gives me a whopping 1 token per second compared to the old GPTQ that gives me a decent 9 tokens per second.. both times I used a same sized model. (I think the slowdown is due to AutoGPTQ using the newer cuda branch which is much slower than the old one)
-
Guanaco 7B, 13B, 33B and 65B models by Tim Dettmers: now for your local LLM pleasure
Are you using a later version of GPTQ-for-LLaMa? If so, go to ooba's CUDA fork (https://github.com/oobabooga/GPTQ-for-LLaMa). That's what I made it in and it definitely works with that. And that's what's included in the one-click-installers.
-
Any idea Vicuna 13B 4bit model output random content?
This usually happens when using models that conflict with your GPTQ installation. You should be using this fork: https://github.com/oobabooga/GPTQ-for-LLaMa. If you did the manual installation wrong, use the one click installer instead.
-
GPT4All: A little helper to get started
cd text-generation-webui # wherever you have it installed mkdir -p repositories cd repositories git clone https://github.com/oobabooga/GPTQ-for-LLaMa -b cuda GPTQ-for-LLaMa cd GPTQ-for-LLaMa python setup_cuda install
- wizard-vicuna-13B • Hugging Face
-
Anyone actually running 30b/65b at reasonably high speed? What's your rig?
I'm on GPTQ for LLaMA folder under repositories says it's pointed at https://github.com/oobabooga/GPTQ-for-LLaMa.git. But I've run through the instructions and also applied the monkey patch to train and apply 4 bit lora which may come into play. No idea.
-
Trying to run TheBloke/vicuna-13B-1.1-GPTQ-4bit-128g with latest GPTQ-for-LLaMa CUDA branch
git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda
langflow
-
News DataStax just bought our startup Langflow
Hey folks I'm the Head of DevRel @ DataStax here and just wanted to share to the HN community that in conjunction with this big acquisition news, the LF team has shipped 1.0-alpha of Langflow.
It's a simple `pip install` and the team would love any and all feedback!
https://github.com/logspace-ai/langflow/
-
Node-based AutoGen with local LLMs inside ComfyUI
You can also check langflow, a node UI for langchain https://github.com/logspace-ai/langflow
- Show HN: Rivet – open-source AI Agent dev env with real-world applications
-
Using Retrieval Augmented Generation to Clear Our GitHub Backlog
There's a few tools out there like AgentGPT (https://github.com/reworkd/AgentGPT, although it's a more conversational interface), and (https://github.com/logspace-ai/langflow) and others. I think most developers definitely prefer a code-first interface though like a library but haven't found one that's great yet. We've used them in the past but didn't have the best experience so would love to hear if anyone has worked with a library they found really flexible.
- Show HN: ChainForge, a visual tool for prompt engineering and LLM evaluation
-
Anyone know how to get LangFlow working with oobabooga?
I found this thread talking about it here: https://github.com/logspace-ai/langflow/issues/263
-
Found a fun little open source project called Flowise. It's a drag & drop UI to build your customized LLM flow using LangchainJS
also check https://github.com/logspace-ai/langflow
-
What exactly is AutoGPT?
AutoGPT is basically a demo of what you can do with Langchain. If you want to play with Langchain in a drag and drop blueprint environment I suggest Langflow
-
Launch HN: Fastgen (YC W23) – Visual Low-Code Back End Builder
Hi, I like this! I'm curious what drove the decision to use the vertical block builder style you chose. I'm partial to node-based editors and have been building things with React Flow recently. LangFlow [1] is a good example, but there's lots of UIs that use a similar interface (e.g. Blender [2] and Unity [3]).
[1] https://github.com/logspace-ai/langflow
[2] https://docs.blender.org/manual/en/3.5/interface/controls/no...
[3] https://unity.com/features/unity-visual-scripting
-
Having fun testing CanvasGPT - a new project launching soon
Here's an open source version that's very similar LangFlow
What are some alternatives?
exllama - A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
Flowise - Drag & drop UI to build your customized LLM flow
koboldcpp - A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
langchain-visualizer - Visualization and debugging tool for LangChain workflows
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
private-gpt - Interact with your documents using the power of GPT, 100% privately, no data leaks
one-click-installers - Simplified installers for oobabooga/text-generation-webui.
Local-LLM-Comparison-Colab-UI - Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.
serge - A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.
SillyTavern - LLM Frontend for Power Users.
SillyTavern - LLM Frontend for Power Users. [Moved to: https://github.com/SillyTavern/SillyTavern]