gpt-llama.cpp VS Dependencies

hi! I really like the solution https://github.com/keldenl/gpt-llama.cpp which helps to deploy https://github.com/mckaywrigley/chatbot-ui on the local model. I am running this together with Wizard7b or 13b locally and it works fine, but when I tried to upload to a remote server I met an error.

sounds like you’re asking for exactly this? https://github.com/keldenl/gpt-llama.cpp

I salut you good sir. This is an amazing idea. I don't have time but it will be interesting idea to use this wrapper https://github.com/keldenl/gpt-llama.cpp which simulates GPT endpoint for local lama, so basically we can have amazing tool for completely free use. If somebody test it please let me know underneath my comment!

I would gladly buy your product to run with a local model, like Vicuna ggml , also see https://github.com/keldenl/gpt-llama.cpp/

possible through fastllama in python or gpt-llama.cpp an API wrapper around llama.cpp

https://github.com/keldenl/gpt-llama.cpp supports embeddings, and it even takes in openai type requests and returns openai compatible responses!

https://github.com/keldenl/gpt-llama.cpp

There's a (kind of) working Auto-GPT solution that uses Vicuna https://github.com/keldenl/gpt-llama.cpp/blob/master/docs/Auto-GPT-setup-guide.md

I did some digging with Dependencies and found that issue is with libstdc++-6.dll

Maybe Dependency Walker can shed some light on that 🤔

I did that, basically. Problem is there is a clblast.dll (on windows) that llama.dll depends on, and it llama-cpp-python always failed dependency resolve to find it. I copied the dll to the right folder, loading it manually via CDLL worked fine, and https://github.com/lucasg/Dependencies also confirmed the dll was findable. When loading DLL's in windows, it checks the same folder for dependency dll's (and a few other places).

You can uses dependencies walker to detect the exact version of MS c++ runtime required.

If that doesn't work you will have to do it the hard way like i did. Using Dependencies to find the missing dlls

("Dependencies" is Dependencies.exe from https://github.com/lucasg/Dependencies)

If you're using MinGW or Cygwin and link in an arbitrary number of system libraries, then you need to ship those files as well. You can use Dependencies to list all DLLs your program is using, including transient dependencies.

A couple of them, yes. Someone else linked Dependencies which is much more modern and doesn't have some of the issues these older applications have. Thank you for the suggestions regardless.