SaaSHub helps you find the best software and product alternatives Learn more →
Web-llm Alternatives
Similar projects and alternatives to web-llm
-
ExpansionCards
Reference designs and documentation to create Expansion Cards for the Framework Laptop
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
-
-
-
-
-
-
-
FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
-
LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
-
-
-
-
ml-ane-transformers
Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE)
-
-
turbopilot
Discontinued Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU
-
-
-
-
gpt-discord-bot
Example Discord bot written in Python that uses the completions API to have conversations with the `text-davinci-003` model, and the moderations API to filter the messages.
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a better web-llm alternative or higher similarity.
web-llm discussion
web-llm reviews and mentions
Posts with mentions or reviews of web-llm.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2024-07-23.
-
Work Stealing: Load-balancing for compute-heavy tasks
As a concrete code example, I recently put out a demo of distributed LLM computing: llama farm, where requests to the website that require llama3 are farmed out to workers. I can run these workers from the command line on a spare laptop, in containers hosted on fly.io, or even from browsers using web-llm. The repo is an implementation of “work stealing,” which enables these llama workers to pull and process jobs at their discretion without exposing a port to http requests or requiring service discovery. Read this post about the implementation, or check out the code.
-
MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
> I'm not sure why they didn't go larger -- having a more efficient model that fits on hardware the size of the RPi could be a gamechanger (IIRC TinyLlama 7B does run, barely).
I'm not sure that RPi is the right target for the next step of local LLMs, and I think that it's worth considering web-deployment on engines like WebLLM [1].
A 7B model may "run fine" on a Raspberry Pi, but I've (personally) found 7B models to be a bit larger than I want to download / run for web-based interfaces.
However, a solid 125M model is the sort of thing that I can run on a webpage, and the time it takes to download to the local user's browser (combined with my bandwidth costs) aren't exorbitant.
[1] https://github.com/mlc-ai/web-llm
-
Ask HN: Browser-Based LLM Models?
check out https://github.com/mlc-ai/web-llm
IMO the main limitation is access to powerful GPUs for running models locally and the size of some models causing UX problems with cold starts
-
Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
Looks like it uses this: https://github.com/mlc-ai/web-llm
- What stack would you recommend to build a LLM app in React without a backend?
-
When LLM doesn’t fit into memory, how to make it work?
So I was playing with MLC webllm locally. I got my mistral 7B model installed and quantised. Converted it using mlc lib to metal package for Apple chips. Now it takes only 3.5GB of memory
-
Show HN: Ollama for Linux – Run LLMs on Linux with GPU Acceleration
Maybe they're talking about https://github.com/mlc-ai/mlc-llm which is used for web-llm (https://github.com/mlc-ai/web-llm)? Seems to be using TVM.
- Local embeddings model for javascript
-
this makes deploying AI language models so much easier
Link to github for those who want to know about MLC straight from them. Web demo is cool but takes a long time to load first time. https://github.com/mlc-ai/web-llm
-
April 2023
web-llm: Bringing large-language models and chat to web browsers. (https://github.com/mlc-ai/web-llm)
-
A note from our sponsor - SaaSHub
www.saashub.com | 3 Oct 2024
Stats
Basic web-llm repo stats
46
12,416
9.4
6 days ago
mlc-ai/web-llm is an open source project licensed under Apache License 2.0 which is an OSI approved license.
The primary programming language of web-llm is TypeScript.
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com