Top 15 Python instruction-tuning Projects

instruction-tuning

• Add a project

• LLaMA-Factory

  4 31,926 9.9 Python

  Efficiently Fine-Tune 100+ LLMs in WebUI (ACL 2024)

  Project mention: Llama-Factory: A WebUI for Efficient Fine-Tuning of 100 LLMs | news.ycombinator.com | 2024-07-17

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• LLaVA

  22 19,655 8.6 Python

  [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

  Project mention: Show HN: LLM Aided OCR (Correcting Tesseract OCR Errors with LLMs) | news.ycombinator.com | 2024-08-09

  This package seems to use llama_cpp for local inference [1] so you can probably use anything supported by that [2]. However, I think it's just passing OCR output for correction - the language model doesn't actually see the original image.

  That said, there are some large language models you can run locally which accept image input. Phi-3-Vision [3], LLaVA [4], MiniCPM-V [5], etc.

  [1] - https://github.com/Dicklesworthstone/llm_aided_ocr/blob/main...

  [2] - https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#de...

  [3] - https://huggingface.co/microsoft/Phi-3-vision-128k-instruct

  [4] - https://github.com/haotian-liu/LLaVA

  [5] - https://github.com/OpenBMB/MiniCPM-V

• LLMSurvey

  3 10,139 4.2 Python

  The official GitHub page for the survey paper "A Survey of Large Language Models".

  

  Project mention: Ask HN: Textbook Regarding LLMs | news.ycombinator.com | 2024-03-23

  Here’s another one - it’s older but has some interesting charts and graphs.

  https://arxiv.org/abs/2303.18223

• self-instruct

  3 4,080 2.3 Python

  Aligning pretrained language models with instruction data generated by themselves.

• Otter

  4 3,560 9.1 Python

  🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

• NExT-GPT

  1 3,241 7.4 Python

  Code and models for NExT-GPT: Any-to-Any Multimodal Large Language Model

• Video-LLaVA

  5 2,888 8.2 Python

  【EMNLP 2024🔥】Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

  

  Project mention: FLaNK Stack Weekly for 27 November 2023 | dev.to | 2023-11-27

• mPLUG-Owl

  2 2,271 6.9 Python

  mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

  

• cambrian

  1 1,713 8.2 Python

  Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

  

  Project mention: Cambrian-1 an Open, Vision-Centric Exploration of Multimodal LLMs | news.ycombinator.com | 2024-06-25

  Code: [cambrian-mllm/*cambrian*](https://github.com/cambrian-mllm/cambrian)

• InternVideo

  3 1,338 8.7 Python

  [ECCV2024] Video Foundation Models & Data for Multimodal Understanding

  

• DataDreamer

  5 813 8.4 Python

  DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models.   🤖💤

  

  Project mention: FLaNK AI - 01 April 2024 | dev.to | 2024-04-01

• DoRA

  1 591 7.9 Python

  [ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation (by NVlabs)

  

  Project mention: FLaNK-AIM Weekly 06 May 2024 | dev.to | 2024-05-06

• HugNLP

  3 374 7.6 Python

  CIKM2023 Best Demo Paper Award. HugNLP is a unified and comprehensive NLP library based on HuggingFace Transformer. Please hugging for NLP now!😊 (by HugAILab)

  

• CodeCapybara

  1 159 5.9 Python

  Open-source Self-Instruction Tuning Code LLM

  

• tasksource

  3 144 6.7 Python

  Datasets collection and preprocessings framework for NLP extreme multitask learning

Python instruction-tuning related posts

• Google Bard AI Now Has the Ability to Understand YouTube Videos

  2 projects | news.ycombinator.com | 24 Nov 2023

• Video-LLaVA

  1 project | /r/hypeurls | 23 Nov 2023

• Video-LLaVA

  3 projects | news.ycombinator.com | 21 Nov 2023

• Share your favorite materials: intersection of LLMs and business applications

  1 project | news.ycombinator.com | 29 Aug 2023

• Recommended open LLMs with image input modality?

  3 projects | /r/LocalLLaMA | 8 Jul 2023

• HugNLP: A Unified and Comprehensive Open-Source Library for NLP

  2 projects | news.ycombinator.com | 3 May 2023

• [R] CodeCapybara: Another open source model for code generation based on instruction tuning, outperformed Llama and CodeAlpaca

  2 projects | /r/MachineLearning | 24 Apr 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 12 Oct 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!