[Project] MLC LLM: Universal LLM Deployment with GPU Acceleration

• mlc-llm

  89 17,053 9.9 Python

  Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.

  

awesome, please share some of your experiences here https://github.com/mlc-ai/mlc-llm/issues/15 if you can, we love to see support for different hws and how well they work. We updated the latest conda so likely you can just install it again

• web-llm

  43 9,300 9.1 TypeScript

  Bringing large-language models and chat to web browsers. Everything runs inside the browser with no server support.

  

It’s pretty smooth to use a ML compiler to target various GPU backends - the project was originally only for WebGPUs (https://mlc.ai/web-llm/), which is around hundreds of lines, and then it only takes tens of lines to re-target it to Vulkan, Metal and CUDA!

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• SillyTavern

  75 677 10.0 JavaScript

  Discontinued LLM Frontend for Power Users. [Moved to: https://github.com/SillyTavern/SillyTavern] (by Cohee1207)

  

Can it be used as a server, though, through an API? We already have powerful frontends like SillyTavern, which can even run on a smartphone, so combining both would be very interesting indeed.

• MIOpen

  9 983 9.7 Assembly

  AMD's Machine Intelligence Library

  

More than three months behind schedule...

Suggest a related project