Mistral 7B Paper on ArXiv

• ollama

  198 62,615 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

• TinyLlama

  14 6,818 8.7 Python

  The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

As discussed in the original GPT3 paper (https://twitter.com/gneubig/status/1286731711150280705?s=20)

TinyLlama is trying to do that for 1.1B: https://github.com/jzhang38/TinyLlama

As long as we are not at the capacity limit, we will have a few of these 7B beats 13B (or 7B beats 70B) moments.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• text-generation-inference

  29 7,881 9.6 Python

  Large Language Model Text Generation Inference

  

A simple microservice would be https://github.com/huggingface/text-generation-inference .

Works flawlessly in Docker on my Windows machine, which is extremely shocking.

• koboldcpp

  180 3,817 10.0 C++

  A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

Koboldcpp has an OpenAI (and kobold api) endpoint now, and supports grammar syntax like you said:

https://github.com/LostRuins/koboldcpp

The biggest catch is it doesn't support llama.cpp's continuous batching yet. Maybe soon?

• text-generation-webui

  876 36,293 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

> Is there some convenience wrapper around this to drop-in replace the OpenAI api with it?

text-generation-webui has an OpenAI API implementation.

> I heard that you could force the model to output JSON even better than ChatGPT with a specific syntax, and that you have to structure the prompts in a certain way to get ok-ish outputs instead of nonsense.

Probably to get the maximum use out of that (particularly the support for grammars), it would be better not to use the OpenAI API implementation, and just use the native API in text-generation-webui (or any other runner for the model that supports grammars or the other features you are looking for.)

[0] https://github.com/oobabooga/text-generation-webui

• godot-llm-experiment

  2 31 7.7 C++

  Getting an LLM to work with Godot.

I put Mistral-7B-Instruct in a Godot poc game the other day, and the simulated conversations it generates is funny as heck:

https://github.com/opyate/godot-llm-experiment

• esther

  3 39 9.6 Clojure

  Dear Esther, you're about to become an idea for a diary app that embeds an LLM.

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project