[GIVEAWAY] Personal GPT: An AI Chatbot that runs with no internet connection on your iPhone

• ggml

  69 9,725 9.8 C

  Tensor library for machine learning

I SFT fine tuned this OSS model and then quantized it down to fit on iOS devices. Also, CoreML isn't very suitable for LLM inference ATM, because you need a KV cache for decoder only LLM (aka GPT) inference, and there isn't a straightforward way to implement it with CoreML. The app initially launched with a GGML based backend, and the grew into a custom fork of it and I'm currently in the process of switching to a completely different, custom backend.

• gorilla

  51 10,118 8.9 Python

  Gorilla: An API store for LLMs

Thanks for the suggestion! The current version is very privacy focused and sandboxed to not connect to the internet, at all. But this is doable in principle. What you're suggesting is the essentially the motivation behind papers like Gorilla, Toolformer, etc.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project