SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Python llama Projects
-
Implementation: ORPO has been integrated into popular fine-tuning libraries like TRL, Axolotl, and LLaMA-Factory.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
vLLM stands for virtual large language models. It is one of the open source fast inferencing and serving libraries. As the name suggests, ‘virtual’ encapsulates the concept of virtual memory and paging from operating systems, which allows addressing the problem of maximum utilization of resources and providing faster token generation by utilizing PagedAttention. Traditional LLM serving involves storing large attention keys and value tensors in GPU memory, leading to inefficient memory usage.
-
LLaVA
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
Project mention: Show HN: LLM Aided OCR (Correcting Tesseract OCR Errors with LLMs) | news.ycombinator.com | 2024-08-09This package seems to use llama_cpp for local inference [1] so you can probably use anything supported by that [2]. However, I think it's just passing OCR output for correction - the language model doesn't actually see the original image.
That said, there are some large language models you can run locally which accept image input. Phi-3-Vision [3], LLaVA [4], MiniCPM-V [5], etc.
[1] - https://github.com/Dicklesworthstone/llm_aided_ocr/blob/main...
[2] - https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#de...
[3] - https://huggingface.co/microsoft/Phi-3-vision-128k-instruct
[4] - https://github.com/haotian-liu/LLaVA
[5] - https://github.com/OpenBMB/MiniCPM-V
-
Hi,
Yes you can. The community creates quantized variants of these that can run on consumer GPUs. A 4-bit quantization of LLAMA 70b works pretty well on Macbook pros, the neural engine with unified CPU memory is quite solid for these. GPUs is a bit tougher because consumer GPU RAM is still kinda small.
You can also fine-tune them. There are lot of frameworks like unsloth that make this easier. https://github.com/unslothai/unsloth . Fine-tuning can be pretty tricky to get right, you need to be aware of things like learning rates, but there are good resources on the internet where a lot of hobbyists have gotten things working. You do not need a PhD in ML to accomplish this. You will, however, need data that you can represent textually.
Source: Director of Engineering for model serving at Databricks.
-
-
Project mention: Generating audiobooks from E-books with Kokoro-82M | news.ycombinator.com | 2025-01-15
-
ChuanhuChatGPT
GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.
-
PaddleNLP
👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
-
Project mention: Show HN: Toolkit for LLM Fine-Tuning, Ablating and Testing | news.ycombinator.com | 2024-04-07
This is a great project, little bit similar to https://github.com/ludwig-ai/ludwig, but it includes testing capabilities and ablation.
questions regarding the LLM testing aspect: How extensive is the test coverage for LLM use cases, and what is the current state of this project area? Do you offer any guarantees, or is it considered an open-ended problem?
Would love to see more progress toward this area!
-
OpenLLM
Run any open-source LLMs, such as Llama, Mistral, as OpenAI compatible API endpoint in the cloud.
OpenLLM is a powerful platform that empowers developers to leverage the potential of open-source large language models (LLMs). It is like a Swiss Army knife for LLMs. It's a set of tools that helps developers overcome these deployment hurdles.
-
shell_gpt
A command-line productivity tool powered by AI large language models like GPT-4, will help you accomplish your tasks faster and more efficiently.
I use a cli tool 100x/day for misc things; can hit openai or a local llm.
https://github.com/TheR1D/shell_gpt
Has nice shell integration.
-
petals
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
Project mention: Serving AI from the Basement – 192GB of VRAM Setup | news.ycombinator.com | 2024-09-08 -
Project mention: Ask HN: What are the drawbacks of caching LLM responses? | news.ycombinator.com | 2024-03-15
Just found this: https://github.com/zilliztech/GPTCache which seems to address this idea/issue.
-
inference
Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
Project mention: GreptimeAI + Xinference - Efficient Deployment and Monitoring of Your LLM Applications | dev.to | 2024-01-24Xorbits Inference (Xinference) is an open-source platform to streamline the operation and integration of a wide array of AI models. With Xinference, you’re empowered to run inference using any open-source LLMs, embedding models, and multimodal models either in the cloud or on your own premises, and create robust AI-driven applications. It provides a RESTful API compatible with OpenAI API, Python SDK, CLI, and WebUI. Furthermore, it integrates third-party developer tools like LangChain, LlamaIndex, and Dify, facilitating model integration and development.
-
-
-
Project mention: Language Models Are Super Mario: Absorbing Abilities from Homologous Models | news.ycombinator.com | 2024-04-06
For others like me who’d not heard of merging before, this seems to be one tool[0] (there may be others)
[0] https://github.com/arcee-ai/mergekit
-
Huatuo-Llama-Med-Chinese
Repo for BenTsao [original name: HuaTuo (华驼)], Instruction-tuning Large Language Models with Chinese Medical Knowledge. 本草（原名：华驼）模型仓库，基于中文医学知识的大语言模型指令微调
-
h2o-llmstudio
H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/
-
InternGPT
InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)
-
Project mention: Understanding the BM25 full text search algorithm | news.ycombinator.com | 2024-11-19
In the Langroid[1] LLM library we have a clean, extensible RAG implementation in the DocChatAgent[2] -- it uses several retrieval techniques, including lexical (bm25, fuzzy search) and semantic (embeddings), and re-ranking (using cross-encoder, reciprocal-rank-fusion) and also re-ranking for diversity and lost-in-the-middle mitigation:
[1] Langroid - a multi-agent LLM framework from CMU/UW-Madison researchers https://github.com/langroid/langroid
[2] DocChatAgent Implementation -
-
Video-LLaMA
[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
-
lightllm
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python llama discussion
Python llama related posts
-
Google and Anthropic are working on AI agents - so I made an open source alternative
-
T2x – a CLI tool for AI-first text operations
-
Llama-3.3-70B-Instruct
-
Fish Speech 1.5
-
Build a Competitive Intelligence Tool Powered by AI
-
Writing an AnythingLLM Custom Agent Skill to Trigger Make.com Webhooks
-
AI Copilot for Data Analysis
-
A note from our sponsor - SaaSHub
www.saashub.com | 15 Jan 2025
Index
What are some of the best open-source llama projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | LLaMA-Factory | 37,832 |
| 2 | vllm | 33,579 |
| 3 | LLaVA | 21,014 |
| 4 | unsloth | 20,203 |
| 5 | Chinese-LLaMA-Alpaca | 18,623 |
| 6 | fish-speech | 18,294 |
| 7 | ChuanhuChatGPT | 15,341 |
| 8 | PaddleNLP | 12,282 |
| 9 | ludwig | 11,272 |
| 10 | OpenLLM | 10,368 |
| 11 | shell_gpt | 10,104 |
| 12 | petals | 9,340 |
| 13 | GPTCache | 7,333 |
| 14 | inference | 5,895 |
| 15 | Baichuan-7B | 5,680 |
| 16 | lmdeploy | 5,137 |
| 17 | mergekit | 5,097 |
| 18 | Huatuo-Llama-Med-Chinese | 4,539 |
| 19 | h2o-llmstudio | 4,097 |
| 20 | InternGPT | 3,213 |
| 21 | langroid | 2,912 |
| 22 | Video-LLaMA | 2,873 |
| 23 | lightllm | 2,746 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com