lm-scorer
📃Language Model based sentences scoring library (by simonepri)
ModuleFormer
ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts. We released a collection of ModuleFormer-based Language Models (MoLM) ranging in scale from 4 billion to 8 billion parameters. (by IBM)
| lm-scorer | ModuleFormer | |
|---|---|---|
| 4 | 1 | |
| 294 | 216 | |
| - | 4.6% | |
| 0.0 | 5.7 | |
| about 2 years ago | 24 days ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
lm-scorer
Posts with mentions or reviews of lm-scorer.
We have used some of these posts to build our list of alternatives
and similar projects.
- How to obbtain probability for entire sequence (Huggingface transformers)
-
MLM vs CLM for actual language modeling
I've tried this once and found the CLM score to be a better indicator than BERT log prob for my use-case. For CLM, I had used lm-scorer.
- "simonepri/lm-scorer: Language Model based sentences scoring library" ("This package provides a simple programming interface to score sentences using different ML language models.")
-
Whole sentence rather than word frequency nltk?
As in, how generally would a sentence make sense in the totality of English? You could look into language models that give probability of a sentence. You can try a library called lm-scorer.
ModuleFormer
Posts with mentions or reviews of ModuleFormer.
We have used some of these posts to build our list of alternatives
and similar projects.
What are some alternatives?
When comparing lm-scorer and ModuleFormer you can also consider the following projects:
transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
LMOps - General technology for enabling AI capabilities w/ LLMs and MLLMs
penney - Penney's Game
StableLM - StableLM: Stability AI Language Models
Sentence-Adder-Anki-Addon - Add sentences to Anki editor window in one click
lingvo - Lingvo
Tyche - A library for probabilistic reasoning and belief modelling in Python.
mixture-of-experts - PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538
tutel - Tutel MoE: An Optimized Mixture-of-Experts Implementation