Personal GPT: A tiny AI Chatbot that runs fully offline on your iPhone

• gpt4all

  139 64,046 9.8 C++

  gpt4all: run open-source LLMs anywhere

  

For PC, please try gpt4all from the fine folks at nomic. For android, there's MLC chat.

• open_llama

  52 7,193 5.3

  OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

  

I can't use Llama or any model from the Llama family, due to license restrictions. Although now there's also the OpenLlama family of models, which have the same architecture but were trained on an open dataset (RedPajama, the same dataset the base model in my app was trained on). I'd love to pursue the direction of extended context lengths for on-device LLMs. Likely in a month or so, when I've implemented all the product feature that I currently have on my backlog.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• RWKV-LM

  84 11,619 8.8 Python

  RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Thanks for the support! Two weeks ago, I'd have said longer contexts on small on-device LLMs are at least a year away, but developments from last week seem to indicate that it's well within reach. Once the low hanging product features are done, I think it's a worthy problem to spend a couple of weeks or perhaps even months on. Speaking of context lengths, recurrent models like RWKV technically have infinite context lengths, but in practice the context slowly fades away after a few thousands of tokens.

• ggml

  69 9,642 9.8 C

  Tensor library for machine learning

Please join r/LocalLLaMA and also look into ggml, llama.cpp.

• Open-Instruction-Generalist

  1 203 6.4 Python

  Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks

  

It's currently based on an SFT tuned version of this model. The SFT dataset (OIG-small-chip2) was mostly around tasks and had no general knowledge in it. So, the base model's knowledge cut-off still holds. And the base model's knowledge cutoff is late 2022, AFAIK.

• gnothi

  10 160 9.3 HTML

  Gnothi is an open-source AI journal and toolkit for self-discovery. If you're interested in getting involved, we'd love to hear from you.

  

I created a GPT journal and am now looking into LocalLLM for the privacy-focused. I gave RedPajama-INCITE a spin, and found it soooo bad! I mean.. really bad. I should note this was about 2 months ago? I did see they released a newer version, I'm assuming there was a data-collection / training process they were waiting on. But I'm wondering: is the magic in the fine-tuning? As in, did you find the model useful before you fine-tuned, and the fine-tuning was just a cherry on top? Or did you have to fine-tune to make it work?

• lila

  794 14,578 10.0 Scala

  ♞ lichess.org: the forever free, adless and open source chess server ♞

  

Lichess for chess. I'm sure you know about it.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• llama.cpp

  769 55,846 10.0 C++

  LLM inference in C/C++

Please join r/LocalLLaMA and also look into ggml, llama.cpp.

• gorilla

  51 10,026 8.9 Python

  Gorilla: An API store for LLMs

Thanks! I agree with your quip about true/proper personal assistants. I'm thinking out loud here. On iOS, accessing and modifying users' calendar and reminders can be done programatically with EventKit. On the LLM side of things, there are techniques to fine tune LLMs to use APIs, like Gorilla, ToolFormer, etc. Perhaps out of scope for Personal GPT, but I suspect it should be possible to build a better (in some aspects) on-device Siri with these primitives.

• RedPajama-Data

  19 4,329 6.0 Python

  The RedPajama-Data repository contains code for preparing large datasets for training large language models.

  

The hallucinations are coming from the LLM interpolating from the training data, substantial portions of which is scraped off of the internet. Because other peoples' prompts never leave their devices (this app makes no internet connections).

Suggest a related project