AnglE
Finetune_LLMs
AnglE | Finetune_LLMs | |
---|---|---|
12 | 2 | |
355 | 438 | |
- | - | |
9.2 | 8.5 | |
about 1 month ago | about 1 month ago | |
Python | Python | |
MIT License | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
AnglE
- FLaNK Stack Weekly 22 January 2024
- Show HN: Sentence Embedding for Vector Search
- UAE: New Sentence Embeddings for RAG | SOTA on MTEB Leaderboard
- [P]UAE: New Sentence Embeddings for RAG | SOTA on MTEB Leaderboard
- [P] UAE: New Sentence Embeddings for RAG | SOTA on MTEB Leaderboard
- Show HN: SOTA Sentence Embeddings on MTEB Leaderboard
Finetune_LLMs
-
Prepare Dataset
Regarding this: if you have resources (at least Colab Pro), you would be much better off training GPT-J (aka GPT-J-6B). Not only it's 4x larger than the largest GPT-2, its architecture, AFAIK, is based on GPT-3. You can use this repo as a good example for GPT-J finetuning.
-
[D] Fine-tuning GPT-J: lessons learned
And this: https://github.com/mallorbc/Finetune_GPTNEO_GPTJ6B
What are some alternatives?
code-llama-for-vscode - Use Code Llama with Visual Studio Code and the Continue extension. A local LLM alternative to GitHub Copilot.
DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
mteb - MTEB: Massive Text Embedding Benchmark
mesh-transformer-jax - Model parallel transformers in JAX and Haiku
api-for-open-llm - Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口
instructor-embedding - [ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings
GoLLIE - Guideline following Large Language Model for Information Extraction
llmware - Providing enterprise-grade LLM-based development framework, tools, and fine-tuned models.
replicate-llama2-sms-chatbot
synthetic-data-generator - 🦄 Use GPT to generate and label data
go-llama2 - Llama 2 inference in one file of pure Go