DeBERTa
BERTweet
| DeBERTa | BERTweet | |
|---|---|---|
| 4 | 1 | |
| 1,853 | 558 | |
| 1.3% | 0.4% | |
| 3.6 | 2.6 | |
| 7 months ago | 5 months ago | |
| Python | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
DeBERTa
-
🚀 Microsoft AI Introduce DeBERTa-V3: A Novel Pre-Training Paradigm for Language Models Based on the Combination of DeBERTa and ELECTRA
Quick Read: https://www.marktechpost.com/2023/03/23/microsoft-ai-introduce-deberta-v3-a-novel-pre-training-paradigm-for-language-models-based-on-the-combination-of-deberta-and-electra/ Paper: https://arxiv.org/pdf/2111.09543.pdf Github: https://github.com/microsoft/DeBERTa
-
How do I fine-tune zero shot models?
The obvious answer is to look at how the Deberta model was trained for MNLI and copy that: github
-
[D] Paper Explained - DeBERTa: Decoding-enhanced BERT with Disentangled Attention (Full Video Analysis)
Code: https://github.com/microsoft/DeBERTa
- DeBERTa-0.9b/1.5b checkpoints released (SuperGLUE score: 89.9%)
BERTweet
-
A pre-trained BERT-like model with recent events?
Not sure if that's what you are looking for, but BERTweet has model trained on tweets containing COVID keywords https://github.com/VinAIResearch/BERTweet
What are some alternatives?
transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
simpletransformers - Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI
clip-as-service - 🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
CoWin-Vaccine-Notifier - Automated Python Script to retrieve vaccine slots availability and get notified when a slot is available.
bertviz - BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)
nlu - 1 line for thousands of State of The Art NLP models in hundreds of languages The fastest and most accurate way to solve text problems.
gector - Official implementation of the papers "GECToR – Grammatical Error Correction: Tag, Not Rewrite" (BEA-20) and "Text Simplification by Tagging" (BEA-21)
kiri - Backprop makes it simple to use, finetune, and deploy state-of-the-art ML models.
sapbert - [NAACL'21 & ACL'21] SapBERT: Self-alignment pretraining for BERT & XL-BEL: Cross-Lingual Biomedical Entity Linking.
covid-19-germany-gae - COVID-19 statistics for Germany. For states and counties. With time series data. Daily updates. Official RKI numbers.
spokestack-python - Spokestack is a library that allows a user to easily incorporate a voice interface into any Python application with a focus on embedded systems.
FARM - :house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.