langflow VS GPTQ-for-LLaMa

Hey folks I'm the Head of DevRel @ DataStax here and just wanted to share to the HN community that in conjunction with this big acquisition news, the LF team has shipped 1.0-alpha of Langflow.

It's a simple `pip install` and the team would love any and all feedback!

https://github.com/logspace-ai/langflow/

You can also check langflow, a node UI for langchain https://github.com/logspace-ai/langflow

There's a few tools out there like AgentGPT (https://github.com/reworkd/AgentGPT, although it's a more conversational interface), and (https://github.com/logspace-ai/langflow) and others. I think most developers definitely prefer a code-first interface though like a library but haven't found one that's great yet. We've used them in the past but didn't have the best experience so would love to hear if anyone has worked with a library they found really flexible.

I found this thread talking about it here: https://github.com/logspace-ai/langflow/issues/263

also check https://github.com/logspace-ai/langflow

AutoGPT is basically a demo of what you can do with Langchain. If you want to play with Langchain in a drag and drop blueprint environment I suggest Langflow

Hi, I like this! I'm curious what drove the decision to use the vertical block builder style you chose. I'm partial to node-based editors and have been building things with React Flow recently. LangFlow [1] is a good example, but there's lots of UIs that use a similar interface (e.g. Blender [2] and Unity [3]).

[1] https://github.com/logspace-ai/langflow

[2] https://docs.blender.org/manual/en/3.5/interface/controls/no...

[3] https://unity.com/features/unity-visual-scripting

Here's an open source version that's very similar LangFlow

git clone https://github.com/oobabooga/GPTQ-for-LLaMa

If you're using the model directly from ehartford, that one hasn't been quantized. Try using the GPTQ quantized version here, and use this fork of GPTQ-for-LLaMa. Load in 4-bit with --wbits 4

This worked for me on Ubuntu. If you want to use the CUDA branch instead of triton, do the same steps except clone this GPTQ-for-LLaMa fork and run python setup_cuda.py install

If you don't have triton and you use AutoGPTQ you're gonna notice a huge slow down compared to the old GPTQ-for-LLaMA cuda branch. For me AutoGPTQ gives me a whopping 1 token per second compared to the old GPTQ that gives me a decent 9 tokens per second.. both times I used a same sized model. (I think the slowdown is due to AutoGPTQ using the newer cuda branch which is much slower than the old one)

Are you using a later version of GPTQ-for-LLaMa? If so, go to ooba's CUDA fork (https://github.com/oobabooga/GPTQ-for-LLaMa). That's what I made it in and it definitely works with that. And that's what's included in the one-click-installers.

This usually happens when using models that conflict with your GPTQ installation. You should be using this fork: https://github.com/oobabooga/GPTQ-for-LLaMa. If you did the manual installation wrong, use the one click installer instead.

cd text-generation-webui # wherever you have it installed mkdir -p repositories cd repositories git clone https://github.com/oobabooga/GPTQ-for-LLaMa -b cuda GPTQ-for-LLaMa cd GPTQ-for-LLaMa python setup_cuda install

I'm on GPTQ for LLaMA folder under repositories says it's pointed at https://github.com/oobabooga/GPTQ-for-LLaMa.git. But I've run through the instructions and also applied the monkey patch to train and apply 4 bit lora which may come into play. No idea.

git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda