EdgeGPT VS dalai

Wallpaper: Forest Stairs Chatbot: Edge GPT Scratchpad: HPR Scratcher(Is not maintained anymore, but still works. The main project is now Pyprland, but stopped working for me currently)

# Read and parse cookies cookies = None if args.cookie_file: cookies = json.loads(open(args.cookie_file, encoding="utf-8").read()) bot = await Chatbot.create(proxy=args.proxy, cookies=cookies) session = _create_session() completer = _create_completer(["!help", "!exit", "!reset"]) initial_prompt = args.prompt https://github.com/acheong08/EdgeGPT/blob/master/src/EdgeGPT.py

Yes, there is this extension: https://github.com/acheong08/EdgeGPT

EdgeGPT extension for Text Generation Webui based on EdgeGPT by acheong08. Now you can give Internet access to your chatbot, easily, quickly and free.

Have a look at this project. Also worth checking out his other repos, such as this one (not sure what exactly is different) or this one for the bing chat.

# Bing Conversation **Human:*\*  I want to simulate inner thinking for a chatbot so I want to link 2 instances of bingchat & make them prompt eachother I am gonna use this unnoficial api https://github.com/acheong08/EdgeGPT to do that, read the documentations for the repo provided and give me code to achieve the mentioned goals *** **Assistant:*\* I’m glad you’re interested in simulating inner thinking for a chatbot. I can help you with some code snippets to use EdgeGPT, a reverse engineered API of Microsoft’s Bing Chat AI[1](https://github.com/acheong08/EdgeGPT).

Hey folks! I'm not sure if it's done before but I just made a discord bot using bing chat on top of open-source EdgeGPT. Used bing chat because it's one of the easiest ways to interact with GPT-4 IMO together with active web browsing. The bot is able to handle /ask and /imagine commands. Happy to have your feedback or issues if you encounter any. Cheers!

I agree, I've definitely seen way more information about running image synthesis models like Stable Diffusion locally than I have LLMs. It's counterintuitive to me that Stable Diffusion takes less RAM than an LLM, especially considering it still needs the word vectors. Goes to show I know nothing.

I guess it comes down to the requirement of a very high end (or multiple) GPU that makes it impractical for most vs just running it in Colab or something.

Tho there are some efforts:

https://github.com/cocktailpeanut/dalai

If you're just looking to play with something locally for the first time, this is the simplest project I've found and has a simple web UI: https://github.com/cocktailpeanut/dalai

It works for 7B/13B/30B/65B LLaMA and Alpaca (fine-tuned LLaMA which definitely works better). The smaller models at least should run on pretty much any computer.

I am not against easy mode options dude, for example I used to run GANs through command line. I replaced them with Upscayl when I found it. Convenience is king after all. Something about this one isn't right though. They are advertising it as a model they built meanwhile their own github show it to be a frontend of LLAMA. Why aren't they honest about it? Why use bots to spam about it? This causes me to not trust the executable they share to 1 to 1 compliation of the source code neither. I would still recommend looking for more decent alternatives. Btw, running it directly isn't that complicated

https://github.com/ggerganov/llama.cpp https://github.com/oobabooga/text-generation-webui https://github.com/mlc-ai/mlc-llm https://github.com/cocktailpeanut/dalai https://github.com/ido-pluto/catai (this is super easy to install but it doesnt provide an api or have integration with langchain)

This was easy to get running: https://github.com/cocktailpeanut/dalai with alpaca 13B (on my 16GB or ram)

I had it running before with Dalai (https://github.com/cocktailpeanut/dalai) but have since moved to using the browser based WebGPU method (https://mlc.ai/web-llm/) which uses Vicuna 7B and is quite good.

https://github.com/nsarrazin/serge let's you pick which model and runs in a container. For API https://github.com/cocktailpeanut/dalai looks super promising.