SimCSE
DiffCSE
| SimCSE | DiffCSE | |
|---|---|---|
| 2 | 3 | |
| 3,255 | 286 | |
| 1.3% | - | |
| 0.0 | 0.0 | |
| 8 months ago | over 1 year ago | |
| Python | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SimCSE
-
BERT-Based Clustering on a Corpus of Genre Samples Kinda Sucks. Why?
Base BERT sentence embeddings are just not good for a couple of reasons and there's some research papers that show this. You can try SimCSE, Google's USE or SBERT as mentioned previously and you'll get better output. It's just an inherent flaw to base BERT that it can't produce good sentence embeddings. Papers have shown you probably will get better scores using GloVe embeddings from scratch than base BERT.
-
State of the Art in Sentence Embeddings
To answer your question about sentence embedding SOTA, it is not s-Bert and hasn't been for a while. SimCSE officially takes the crown since it's been presented at a conference, though according to paperswithcode's benchmark leaderboard there are other papers on arxiv that report higher performance on STS and similar tasks such as DCPCSE. Having tried both of these for my use case I found SimCSE to be better but YMMV.
DiffCSE
-
[N] MIT/Meta AI released their new SOTA unsupervised sentence embedding model "DiffCSE"
Code for https://arxiv.org/abs/2204.10298 found: https://github.com/voidism/DiffCSE
What are some alternatives?
PromCSE - Code for "Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learning (EMNLP 2022)"
inltk - Natural Language Toolkit for Indic Languages aims to provide out of the box support for various NLP tasks that an application developer might need
Revisiting-Contrastive-SSL - Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations. [NeurIPS 2021]
AnnA_Anki_neuronal_Appendix - Using machine learning on your anki collection to enhance the scheduling via semantic clustering and semantic similarity
Unsupervised-Classification - SCAN: Learning to Classify Images without Labels, incl. SimCLR. [ECCV 2020]
BERTopic - Leveraging BERT and c-TF-IDF to create easily interpretable topics.
Unsupervised-Semantic-Segmentation - Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals. [ICCV 2021]
pytorch-metric-learning - The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.