Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 23 llama2 Open-Source Projects
-
-
jan
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
LLaVA
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
-
h2ogpt
Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://codellama.h2o.ai/
-
petals
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
-
-
h2o-llmstudio
H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://h2oai.github.io/h2o-llmstudio/
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
opencompass
OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
-
casibase
⚡️Open-source AI LangChain-like RAG (Retrieval-Augmented Generation) knowledge database with web UI and Enterprise SSO⚡️, supports OpenAI, Azure, LLaMA, Google Gemini, HuggingFace, Claude, Grok, etc., chat bot demo: https://demo.casibase.com, admin UI demo: https://demo-admin.casibase.com
-
api-for-open-llm
Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口
-
-
cortex
Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM). Powers 👋 Jan (by janhq)
-
DemoGPT
Create 🦜️🔗 LangChain apps by just using prompts🌟 Star to support our work! | 只需使用句子即可创建 LangChain 应用程序。 给个star支持我们的工作吧！
-
-
enchanted
Enchanted is iOS and macOS app for chatting with private self hosted language models such as Llama2, Mistral or Vicuna using Ollama.
-
-
-
Get-Things-Done-with-Prompt-Engineering-and-LangChain
LangChain & Prompt Engineering tutorials on Large Language Models (LLMs) such as ChatGPT with custom data. Jupyter notebooks on loading and indexing data, creating prompt templates, CSV agents, and using retrieval QA chains to query the custom data. Projects for using a private LLM (Llama 2) for chat with PDF files, tweets sentiment analysis.
-
-
distributed-llama
Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: OpenInterpreter – Natural language interface to your computer | news.ycombinator.com | 2024-04-23
As we continue this blog series, let's explore a fully open-source alternative to LM Studio - Jan, a project from Southeast Asia.
Project mention: Show HN: I Remade the Fake Google Gemini Demo, Except Using GPT-4 and It's Real | news.ycombinator.com | 2023-12-10Update: For anyone else facing the commercial use question on LLaVA - it is licensed under Apache 2.0. Can be used commercially with attribution: https://github.com/haotian-liu/LLaVA/blob/main/LICENSE
Project mention: Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023? | news.ycombinator.com | 2023-12-24As others have said you want RAG.
The most feature complete implementation I've seen is h2ogpt[0] (not affiliated).
The code is kind of a mess (most of the logic is in an ~8000 line python file) but it supports ingestion of everything from YouTube videos to docx, pdf, etc - either offline or from the web interface. It uses langchain and a ton of additional open source libraries under the hood. It can run directly on Linux, via docker, or with one-click installers for Mac and Windows.
It has various model hosting implementations built in - transformers, exllama, llama.cpp as well as support for model serving frameworks like vLLM, HF TGI, etc or just OpenAI.
You can also define your preferred embedding model along with various other parameters but I've found the out of box defaults to be pretty sane and usable.
[0] - https://github.com/h2oai/h2ogpt
So how long until we can do an open source Mistral Large?
We could make a start on Petals or some other open source distributed training network cluster possibly?
[0] https://petals.dev/
Project mention: Paid dev gig: develop a basic LLM PEFT finetuning utility | /r/LocalLLaMA | 2023-06-02
Project mention: Show HN: Times faster LLM evaluation with Bayesian optimization | news.ycombinator.com | 2024-02-13Fair question.
Evaluate refers to the phase after training to check if the training is good.
Usually the flow goes training -> evaluation -> deployment (what you called inference). This project is aimed for evaluation. Evaluation can be slow (might even be slower than training if you're finetuning on a small domain specific subset)!
So there are [quite](https://github.com/microsoft/promptbench) [a](https://github.com/confident-ai/deepeval) [few](https://github.com/openai/evals) [frameworks](https://github.com/EleutherAI/lm-evaluation-harness) working on evaluation, however, all of them are quite slow, because LLM are slow if you don't have infinite money. [This](https://github.com/open-compass/opencompass) one tries to speed up by parallelizing on multiple computers, but none of them takes advantage of the fact that many evaluation queries might be similar and all try to evaluate on all given queries. And that's where this project might come in handy.
Project mention: Open-source AI knowledge database with web UI and Enterprise SSO | news.ycombinator.com | 2023-12-21
Another thing to try is one of the repositories like SolidGPT: https://github.com/AI-Citizen/SolidGPT
Jan incorporates a lightweight, built-in inference server called Nitro. Nitro supports both llama.cpp and NVIDIA's TensorRT-LLM engines. This means many open LLMs in the GGUF format are supported. Jan's Model Hub is designed for easy installation of pre-configured models but it also allows you to install virtually any model from Hugging Face or even your own.
Project mention: A comprehensive guide to building RAG-based LLM applications for production | news.ycombinator.com | 2023-10-25
Project mention: RefactAI: Use best-in-class LLMs for coding in your IDE | news.ycombinator.com | 2024-04-30
Nice article. We did a demo for booting to LLM and also as Kernel Module: https://github.com/trholding/llama2.c The whole things was funny and buggy, but since then we have been developing in stealth, even trying to raise VC capital. Our goal is to make computers like a buddy to whom you can talk to and explain things and get work done, kinda like a Jarvis. The way we interact with computers haven't changed for decades, its time to disrupt that to get more productivity. I also believe with this approach one can avoid installing different applications, when the computer (models) emulate activities done through applications. For example, cutting and pasting a dog from a dog photo onto a banner for a dog racing competition would not require you to be a graphics artist nor use tools like photshop / gimp. You could tell the computer and it would use segment anything to cut the dog, use Text and SD for banner text and bg paste the dog, seek your approval, search for the fastest, best and cheapest banner printing service and submit it. 10 years ago this could have been sci-fi, but now it is a possibility. Just need to connect the dots, package and polish it to make it a good product.
Project mention: What AI assistants are already bundled for Linux? | news.ycombinator.com | 2024-03-01Perhaps this: https://github.com/yusufcanb/tlm?
it is not distro bundled (yet), but I have it running on my Fedora Linux 39 running on a NUC with 16GB of RAM. Performance is good enough for me.
Project mention: Get-Things-Done-with-Prompt-Engineering-and-LangChain: NEW Data - star count:617.0 | /r/algoprojects | 2023-12-10
Project mention: feed pdf files into an LLM for question answering tasks | /r/LocalLLaMA | 2023-11-08IYH use chatd
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
llama2 related posts
-
Introducing Jan
-
Limitless: Personalized AI powered by what you've seen, said, and heard
-
Distributed Grok-1 (314B)
-
AI enthusiasm - episode #2🚀
-
Do you Know! Llama ?
-
Ask HN: What is the current (Apr. 2024) gold standard of running an LLM locally?
-
Half-Quadratic Quantization of Large Machine Learning Models
-
A note from our sponsor - InfluxDB
www.influxdata.com | 7 May 2024
Index
What are some of the best open-source llama2 projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | open-interpreter | 48,604 |
| 2 | jan | 17,877 |
| 3 | LLaVA | 16,333 |
| 4 | h2ogpt | 10,458 |
| 5 | petals | 8,684 |
| 6 | Baichuan2 | 3,936 |
| 7 | h2o-llmstudio | 3,602 |
| 8 | opencompass | 2,559 |
| 9 | casibase | 2,151 |
| 10 | api-for-open-llm | 1,999 |
| 11 | SolidGPT | 1,945 |
| 12 | cortex | 1,600 |
| 13 | DemoGPT | 1,573 |
| 14 | llm-applications | 1,504 |
| 15 | enchanted | 1,579 |
| 16 | refact | 1,422 |
| 17 | llama2.c | 1,385 |
| 18 | LLMCompiler | 1,083 |
| 19 | tlm | 1,041 |
| 20 | Get-Things-Done-with-Prompt-Engineering-and-LangChain | 958 |
| 21 | autollm | 914 |
| 22 | chatd | 797 |
| 23 | distributed-llama | 756 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com