ovos-core VS dalai

Check out OVOS.

I used to work for Mycroft, so I'm hoping to eventually create an image that's compatible with Home Assistant pipelines.

For now, though, you may want to check out OVOS: https://openvoiceos.com/

Blast you're so right! I already ordered an M92, but when LLaMA takes off for voice assistants and I max out my RAM I shall have a think about that HP G2!! With one of those maybe I could have enough RAM to run openbsd and have several VMs running on it! Thank you :D

The upper NUC is running a manual install of ovos-core with sound output through the 22" touchscreen monitor and a kinect V1 as a microphone.

Good news, everyone! As those of you who are following OpenVoiceOS GoFundMe campaign may already have spotted, we've surpassed our most recent fundraising target, and set a new one. Our current stretch goal is a doozy. We're looking to give the Assistant a personality! More specifically, a configurable personality, to make your Assistant that much more... yours.

However it did not work for me out-of-the-box. I had to create some custom Dockerfiles to make adjustments to some of the images before all of them would start up correctly. i got around the Rapidfuzz issue you ran I to using the fix described here: https://github.com/OpenVoiceOS/ovos-core/issues/267

OVOS : There's a Dockerfile in the core repo that you should be able to use to make a Docker image for headless OVOS-Mycroft. You'd probably still have to set up the personal backend for it - not sure as I haven't set it up yet. This is likely the voice assistant replacement route that I'm going with, when there is time to do it.

I agree, I've definitely seen way more information about running image synthesis models like Stable Diffusion locally than I have LLMs. It's counterintuitive to me that Stable Diffusion takes less RAM than an LLM, especially considering it still needs the word vectors. Goes to show I know nothing.

I guess it comes down to the requirement of a very high end (or multiple) GPU that makes it impractical for most vs just running it in Colab or something.

Tho there are some efforts:

https://github.com/cocktailpeanut/dalai

If you're just looking to play with something locally for the first time, this is the simplest project I've found and has a simple web UI: https://github.com/cocktailpeanut/dalai

It works for 7B/13B/30B/65B LLaMA and Alpaca (fine-tuned LLaMA which definitely works better). The smaller models at least should run on pretty much any computer.

I am not against easy mode options dude, for example I used to run GANs through command line. I replaced them with Upscayl when I found it. Convenience is king after all. Something about this one isn't right though. They are advertising it as a model they built meanwhile their own github show it to be a frontend of LLAMA. Why aren't they honest about it? Why use bots to spam about it? This causes me to not trust the executable they share to 1 to 1 compliation of the source code neither. I would still recommend looking for more decent alternatives. Btw, running it directly isn't that complicated

https://github.com/ggerganov/llama.cpp https://github.com/oobabooga/text-generation-webui https://github.com/mlc-ai/mlc-llm https://github.com/cocktailpeanut/dalai https://github.com/ido-pluto/catai (this is super easy to install but it doesnt provide an api or have integration with langchain)

This was easy to get running: https://github.com/cocktailpeanut/dalai with alpaca 13B (on my 16GB or ram)

I had it running before with Dalai (https://github.com/cocktailpeanut/dalai) but have since moved to using the browser based WebGPU method (https://mlc.ai/web-llm/) which uses Vicuna 7B and is quite good.

https://github.com/nsarrazin/serge let's you pick which model and runs in a container. For API https://github.com/cocktailpeanut/dalai looks super promising.