llm
serge
llm | serge | |
---|---|---|
23 | 40 | |
2,991 | 5,553 | |
- | 0.9% | |
9.4 | 9.8 | |
5 days ago | 6 days ago | |
Python | Svelte | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
llm
- FLaNK AI-April 22, 2024
-
Show HN: I made a tool to clean and convert any webpage to Markdown
That's a great use case, you might be able to do this if you've got a copy and paste on the command line with
https://github.com/simonw/llm
In between. An alias like pdfwtf translating to "paste | llm command | copy"
-
Command R+: A Scalable LLM Built for Business
I added support for this model to my LLM CLI tool via a new plugin: https://github.com/simonw/llm-command-r
So now you can do this:
pipx install llm
-
The Next Generation of Claude (Claude 3)
If you're willing to use the CLI, Simon Willison's llm library[0] should do the trick.
[0] https://github.com/simonw/llm
- Show HN: I made an app to use local AI as daily driver
-
Localllm lets you develop gen AI apps on local CPUs
I'm not thrilled about https://github.com/GoogleCloudPlatform/localllm/blob/main/ll... calling their Python package "llm" and installing "llm" as a CLI command, when my similar https://llm.datasette.io/ project has that namespace reserved on PyPI already: https://pypi.org/project/llm/
- FLaNK 15 Jan 2024
- Show HN: Simple Script for Enhanced LLM Interaction in Vim
-
Bash One-Liners for LLMs
I've been gleefully exploring the intersection of LLMs and CLI utilities for a few months now - they are such a great fit for each other! The unix philosophy of piping things together is a perfect fit for how LLMs work.
I've mostly been exploring this with my https://llm.datasette.io/ CLI tool, but I have a few other one-off tools as well: https://github.com/simonw/blip-caption and https://github.com/simonw/ospeak
I'm puzzled that more people aren't loudly exploring this space (LLM+CLI) - it's really fun.
-
Semantic Kernel
Seems nice if you're using c# or java. It also supports python, but for that Simon's llm library is nice because he designed it as both a library and a command line tool: https://github.com/simonw/llm
serge
- Show HN: I made an app to use local AI as daily driver
- chatgpt alternative
-
Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2
Very cool, this looks like a combination of chatbot-ui and llama-cpp-python? A similar project I've been using is https://github.com/serge-chat/serge. Nous-Hermes-Llama2-13b is my daily driver and scores high on coding evaluations (https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul...).
-
LeCun: Qualcomm working with Meta to run Llama-2 on mobile devices
You might be pleased to hear that nothing really stops you from doing this today. If you ran Serge[0] on a Mac with Tailscale, you could hack together a decently-accelerated Llama chatbot.
[0] https://github.com/serge-chat/serge
-
Chatbot frontend library in Svelte?
Cannot help you with libraries specifically but both Serge and ChatUI are built using SvelteKit, so the code might be of some use to you.
- We’re back and…
-
Best way to use AMD CPU and GPU
Serge made it really easy for me to get started, but it all CPU-based.
-
Need Help
All that said this project probably solves your problem: https://github.com/serge-chat/serge
- Are you selfhosting a ChatGPT alternative?
-
What the hell??
You can play a little bit with more straightforward local models (the simplest to setup is https://github.com/nsarrazin/serge ), to see that any LLM is basically a party trick.
What are some alternatives?
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
gpt4all - gpt4all: run open-source LLMs anywhere
langroid - Harness LLMs with Multi-Agent Programming
langflow - ⛓️ Langflow is a dynamic graph where each node is an executable unit. Its modular and interactive design fosters rapid experimentation and prototyping, pushing hard on the limits of creativity.
exllama - A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
llama.cpp - LLM inference in C/C++
multi-gpt - A Clojure interface into the GPT API with advanced tools like conversational memory, task management, and more
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
jehuty - Fluent API to interact with chat based GPT model
FastChat - An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
llm-replicate - LLM plugin for models hosted on Replicate
llama-gpt - A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!