Mixtral: Mixture of Experts

• ollama-webui

  14 5,789 9.8 Svelte

  Discontinued ChatGPT-Style WebUI for LLMs (Formerly Ollama WebUI) [Moved to: https://github.com/open-webui/open-webui]

  

On mac silicon:

https://ollama.ai/

ollama pull mixtral

For a chatgpt-esk web ui

https://github.com/ollama-webui/ollama-webui

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main

Navigate to http://localhost:3000

You can also use ollama in langchain.

• ollama

  195 58,943 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

On mac silicon:

https://ollama.ai/

ollama pull mixtral

For a chatgpt-esk web ui

https://github.com/ollama-webui/ollama-webui

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main

Navigate to http://localhost:3000

You can also use ollama in langchain.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• CogVLM

  16 4,968 9.0 Python

  a state-of-the-art-level open visual language model | 多模态预训练模型

  

CogVLM is very good in my (brief) testing: https://github.com/THUDM/CogVLM

The model weights seem to be under a non-commercial license, not true open source, but it is "open access" as you requested.

• llamafile

  34 13,765 9.6 C++

  Distribute and run LLMs with a single file.

  

• LLMstudio

  1 197 9.3 TypeScript

  Framework to bring LLM applications to production

  

Lmstudio (that they linked) is definitely not open source, and doesn't even offer a pricing model for business use.

Llmstudio is, but I suspect that was a typo in their comment. https://github.com/TensorOpsAI/LLMStudio

• llama.cpp

  769 56,891 10.0 C++

  LLM inference in C/C++

> The output quality is not "ruined" at all.

That was my experience as well.

I also tried 2-bit version, and it was horrible.

However, there is a new approach in the works[1] (merged yesterday) which works surprisingly well with 2.10 bits per weight (12.3 GB model size)

[1] https://github.com/ggerganov/llama.cpp/pull/4773

Suggest a related project