Show HN: OfflineLLM – a Vision Pro app running TinyLlama on device

• fllama

  1 44 9.6 C++

  llama.cpp for Flutter

  

Thanks for asking: Yes I did make it, but, no app tying it all together. At least, it isn't out yet.

The grunt work of getting it running on different platforms + nice easy OpenAI compatible interfaces x RAG x voice assistant is open source:

- FLLAMA: https://github.com/Telosnex/fllama

• fonnx

  1 151 9.2 Dart

  ONNX runtime for Flutter.

  

llama.cpp at core, openai compatible API, function call support, multimodal model support, Metal support. All platforms incl. web, but WASM is slow, def. not worth it except as a proof of concept.

- FONNX: https://github.com/Telosnex/fonnx

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project