DeBERTa
sapbert
| DeBERTa | sapbert | |
|---|---|---|
| 4 | 1 | |
| 1,847 | 156 | |
| 1.0% | 1.3% | |
| 3.6 | 0.0 | |
| 7 months ago | about 1 year ago | |
| Python | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
DeBERTa
-
๐ Microsoft AI Introduce DeBERTa-V3: A Novel Pre-Training Paradigm for Language Models Based on the Combination of DeBERTa and ELECTRA
Quick Read: https://www.marktechpost.com/2023/03/23/microsoft-ai-introduce-deberta-v3-a-novel-pre-training-paradigm-for-language-models-based-on-the-combination-of-deberta-and-electra/ Paper: https://arxiv.org/pdf/2111.09543.pdf Github: https://github.com/microsoft/DeBERTa
-
How do I fine-tune zero shot models?
The obvious answer is to look at how the Deberta model was trained for MNLI and copy that: github
-
[D] Paper Explained - DeBERTa: Decoding-enhanced BERT with Disentangled Attention (Full Video Analysis)
Code: https://github.com/microsoft/DeBERTa
- DeBERTa-0.9b/1.5b checkpoints released (SuperGLUE score: 89.9%)
sapbert
-
Weighting embedding similarity by frequency/saliency
For example, when using concept embeddings, I'd like to be able to score "Exam" <-> "Exam" as less important than "Diabetes" <-> "High blood sugar". But obviously, the former has a similarity of 1.
What are some alternatives?
transformers - ๐ค Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
haystack - :mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
clip-as-service - ๐ Scalable embedding, reasoning, ranking for images and sentences with CLIP
pytorch-metric-learning - The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.
BERTweet - BERTweet: A pre-trained language model for English Tweets (EMNLP-2020)
bertviz - BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)
gector - Official implementation of the papers "GECToR โ Grammatical Error Correction: Tag, Not Rewrite" (BEA-20) and "Text Simplification by Tagging" (BEA-21)
spokestack-python - Spokestack is a library that allows a user to easily incorporate a voice interface into any Python application with a focus on embedded systems.
PaddleHelix - Bio-Computing Platform Featuring Large-Scale Representation Learning and Multi-Task Deep Learning โ่บๆๆกจโ็็ฉ่ฎก็ฎๅทฅๅ ท้