Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

• ollama

  209 64,536 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

Streaming is not a problem (it's just a simple flag: https://github.com/wiktor-k/llama-chat/blob/main/index.ts#L2...) but I've never used voice input.

The examples show image input though: https://github.com/ollama/ollama/blob/main/docs/api.md#reque...

Maybe you can file an issue here: https://github.com/ollama/ollama/issues

• llama-chat

  2 6 4.3 TypeScript

  Implements a simple REPL chat with a locally running instance of Ollama. (by wiktor-k)

Streaming is not a problem (it's just a simple flag: https://github.com/wiktor-k/llama-chat/blob/main/index.ts#L2...) but I've never used voice input.

The examples show image input though: https://github.com/ollama/ollama/blob/main/docs/api.md#reque...

Maybe you can file an issue here: https://github.com/ollama/ollama/issues

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• cloudseeder

  5 46 3.6 Shell

  One-click install internet appliances that operate on your terms. Transform your home computer into a sovereign and secure cloud.

  

Ollama is really organized - it relies on llama but the UX and organization it provides makes it legit. We recently made a one-click wizard to run Open WebUI and Ollama together, self hosted and remotely accessible but locally hosted [1]

[1] https://github.com/ipv6rslimited/cloudseeder

• mlx

  23 14,456 9.8 C++

  MLX: An array framework for Apple silicon

  

Yes, we are also looking at integrating MLX [1] which is optimized for Apple Silicon and built by an incredible team of individuals, a few of which were behind the original Torch [2] project. There's also TensorRT-LLM [3] by Nvidia optimized for their recent hardware.

All of this of course acknowledging that llama.cpp is an incredible project with competitive performance and support for almost any platform.

[1] https://github.com/ml-explore/mlx

[2] https://en.wikipedia.org/wiki/Torch_(machine_learning)

[3] https://github.com/NVIDIA/TensorRT-LLM

• TensorRT-LLM

  14 6,705 8.4 C++

  TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

  

Yes, we are also looking at integrating MLX [1] which is optimized for Apple Silicon and built by an incredible team of individuals, a few of which were behind the original Torch [2] project. There's also TensorRT-LLM [3] by Nvidia optimized for their recent hardware.

All of this of course acknowledging that llama.cpp is an incredible project with competitive performance and support for almost any platform.

[1] https://github.com/ml-explore/mlx

[2] https://en.wikipedia.org/wiki/Torch_(machine_learning)

[3] https://github.com/NVIDIA/TensorRT-LLM

• promptfoo

  5 328 10.0 TypeScript

  Discontinued Test your prompts. Evaluate and compare LLM outputs, catch regressions, and improve prompt quality. [Moved to: https://github.com/promptfoo/promptfoo] (by typpo)

  

Jumping in because I'm a big believer in (1) local LLMs, and (2) evals specific to individual use cases.

[0] https://github.com/typpo/promptfoo

• ollama_local_rag

  1 11 5.6 Python

I love working with Ollama, I was really surprised at how easy it is to build a simple RAG system with it. For example: https://github.com/stephen37/ollama_local_rag

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• llama-cpp-python

  55 6,579 9.8 Python

  Python bindings for llama.cpp

There's a Python binding for llama.cpp which is actively maintained and has worked well for me: https://github.com/abetlen/llama-cpp-python

Suggest a related project