DeBERTa
PaddleHelix
| DeBERTa | PaddleHelix | |
|---|---|---|
| 4 | 1 | |
| 1,853 | 789 | |
| 1.3% | 1.1% | |
| 3.6 | 5.4 | |
| 7 months ago | about 2 months ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
DeBERTa
-
🚀 Microsoft AI Introduce DeBERTa-V3: A Novel Pre-Training Paradigm for Language Models Based on the Combination of DeBERTa and ELECTRA
Quick Read: https://www.marktechpost.com/2023/03/23/microsoft-ai-introduce-deberta-v3-a-novel-pre-training-paradigm-for-language-models-based-on-the-combination-of-deberta-and-electra/ Paper: https://arxiv.org/pdf/2111.09543.pdf Github: https://github.com/microsoft/DeBERTa
-
How do I fine-tune zero shot models?
The obvious answer is to look at how the Deberta model was trained for MNLI and copy that: github
-
[D] Paper Explained - DeBERTa: Decoding-enhanced BERT with Disentangled Attention (Full Video Analysis)
Code: https://github.com/microsoft/DeBERTa
- DeBERTa-0.9b/1.5b checkpoints released (SuperGLUE score: 89.9%)
PaddleHelix
-
Baidu and BioMap AI Research Open-Sources HelixFold-Single: An End-To-End MSA-Free Protein Structure Prediction Pipeline
Continue reading | Check out the paper and code.
What are some alternatives?
transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
transferlearning - Transfer learning / domain adaptation / domain generalization / multi-task learning etc. Papers, codes, datasets, applications, tutorials.-迁移学习
clip-as-service - 🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
dipy - DIPY is the paragon 3D/4D+ imaging library in Python. Contains generic methods for spatial normalization, signal processing, machine learning, statistical analysis and visualization of medical images. Additionally, it contains specialized methods for computational anatomy including diffusion, perfusion and structural imaging.
BERTweet - BERTweet: A pre-trained language model for English Tweets (EMNLP-2020)
typedb-ml - TypeDB-ML is the Machine Learning integrations library for TypeDB
bertviz - BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)
eirli - An Empirical Investigation of Representation Learning for Imitation (EIRLI), NeurIPS'21
gector - Official implementation of the papers "GECToR – Grammatical Error Correction: Tag, Not Rewrite" (BEA-20) and "Text Simplification by Tagging" (BEA-21)
Unsupervised-Classification - SCAN: Learning to Classify Images without Labels, incl. SimCLR. [ECCV 2020]
sapbert - [NAACL'21 & ACL'21] SapBERT: Self-alignment pretraining for BERT & XL-BEL: Cross-Lingual Biomedical Entity Linking.
spokestack-python - Spokestack is a library that allows a user to easily incorporate a voice interface into any Python application with a focus on embedded systems.