GPU-Accelerated LLM on a $100 Orange Pi

• relax

  1 129 9.9 Python
  

Its not supported by TVM yet, but there is support for Qualcomm Hexagon.

You can kinda see some of the supported backends gated behind flags in the cmake file: https://github.com/mlc-ai/relax/blob/mlc/CMakeLists.txt

• private-gpt

  131 51,882 9.2 Python

  Interact with your documents using the power of GPT, 100% privately, no data leaks

  

> a local wikipedia dump

There exists (at least) a project to train an LLM on local documents: privateGPT - https://github.com/imartinez/privateGPT

It should lead to the content source, to check the exact text:

> You'll need to wait 20-30 seconds (depending on your machine) while the LLM model consumes the prompt and prepares the answer. Once done, it will print the answer and the 4 sources it used as context from your documents

You will have noticed, in that first sentence, that it may not be practical, especially on an Orange Pi.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• web-stable-diffusion

  21 3,440 4.4 Jupyter Notebook

  Bringing stable diffusion models to web browsers. Everything runs inside the browser with no server support.

  

Yup, here's their web stable diffusion repo: https://github.com/mlc-ai/web-stable-diffusion

The input is a model (weights + runtime lib) compiled via the mlc-llm project: https://mlc.ai/mlc-llm/docs/compilation/compile_models.html

Suggest a related project