Personal GPT: A tiny AI Chatbot that runs fully offline on your iPhone

This page summarizes the projects mentioned and recommended in the original post on /r/ChatGPT

Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers
Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.
www.nutrient.io
featured
CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai
featured
  1. gpt4all

    GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

    For PC, please try gpt4all from the fine folks at nomic. For android, there's MLC chat.

  2. Nutrient

    Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.

    Nutrient logo
  3. open_llama

    OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

    I can't use Llama or any model from the Llama family, due to license restrictions. Although now there's also the OpenLlama family of models, which have the same architecture but were trained on an open dataset (RedPajama, the same dataset the base model in my app was trained on). I'd love to pursue the direction of extended context lengths for on-device LLMs. Likely in a month or so, when I've implemented all the product feature that I currently have on my backlog.

  4. RWKV-LM

    RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

    Thanks for the support! Two weeks ago, I'd have said longer contexts on small on-device LLMs are at least a year away, but developments from last week seem to indicate that it's well within reach. Once the low hanging product features are done, I think it's a worthy problem to spend a couple of weeks or perhaps even months on. Speaking of context lengths, recurrent models like RWKV technically have infinite context lengths, but in practice the context slowly fades away after a few thousands of tokens.

  5. ggml

    Tensor library for machine learning

    Please join r/LocalLLaMA and also look into ggml, llama.cpp.

  6. Open-Instruction-Generalist

    Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks

    It's currently based on an SFT tuned version of this model. The SFT dataset (OIG-small-chip2) was mostly around tasks and had no general knowledge in it. So, the base model's knowledge cut-off still holds. And the base model's knowledge cutoff is late 2022, AFAIK.

  7. gnothi

    Gnothi is an open-source AI journal and toolkit for self-discovery. If you're interested in getting involved, we'd love to hear from you.

    I created a GPT journal and am now looking into LocalLLM for the privacy-focused. I gave RedPajama-INCITE a spin, and found it soooo bad! I mean.. really bad. I should note this was about 2 months ago? I did see they released a newer version, I'm assuming there was a data-collection / training process they were waiting on. But I'm wondering: is the magic in the fine-tuning? As in, did you find the model useful before you fine-tuned, and the fine-tuning was just a cherry on top? Or did you have to fine-tune to make it work?

  8. lila

    ♞ lichess.org: the forever free, adless and open source chess server ♞

    Lichess for chess. I'm sure you know about it.

  9. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  10. llama.cpp

    LLM inference in C/C++

    Please join r/LocalLLaMA and also look into ggml, llama.cpp.

  11. gorilla

    Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

    Thanks! I agree with your quip about true/proper personal assistants. I'm thinking out loud here. On iOS, accessing and modifying users' calendar and reminders can be done programatically with EventKit. On the LLM side of things, there are techniques to fine tune LLMs to use APIs, like Gorilla, ToolFormer, etc. Perhaps out of scope for Personal GPT, but I suspect it should be possible to build a better (in some aspects) on-device Siri with these primitives.

  12. RedPajama-Data

    The RedPajama-Data repository contains code for preparing large datasets for training large language models.

    The hallucinations are coming from the LLM interpolating from the training data, substantial portions of which is scraped off of the internet. Because other peoples' prompts never leave their devices (this app makes no internet connections).

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Lichess: Over 2023, concurrent players online have been higher than the lockdown and Queen's Gambit boom; servers stable so far (@agadmator knows how to take us down!) - Check out all our features, 100% free to everyone: https://lichess.org/features no paywalls, no ads, 100% for love of chess

    2 projects | /r/chess | 22 Jan 2023
  • The Q&A Megathread for new and beginner chess players

    2 projects | /r/chess | 13 Jan 2023
  • Why isn't LiChess employing the concept of "safe move" in analysis?

    2 projects | /r/chess | 21 Nov 2022
  • Becoming a Chess Grandmaster

    3 projects | news.ycombinator.com | 26 Jul 2021
  • Ask HN: Is anybody building an alternative transformer?

    3 projects | news.ycombinator.com | 14 Feb 2025

Did you know that Python is
the 2nd most popular programming language
based on number of references?