bigscience
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data. (by bigscience-workshop)
t-zero
Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization) (by bigscience-workshop)
bigscience | t-zero | |
---|---|---|
4 | 2 | |
939 | 448 | |
0.7% | 2.2% | |
3.2 | 0.0 | |
6 months ago | over 1 year ago | |
Shell | Python | |
GNU General Public License v3.0 or later | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
bigscience
Posts with mentions or reviews of bigscience.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-03-15.
-
[N] Live and open training of BigScience's 176B multilingual language model has just started
Details on the distributed setup used for the training: https://github.com/bigscience-workshop/bigscience/tree/master/train/tr11-176B-ml
-
[Announcement] HuggingFace BigScience AMA Thursday, March 24th from 5pm CET
Model architecture and a blog post on decisions on architecture, size, shape, and pretraining duration
- Where can I see what languages the dataset consists of, which recently started training an open-source model 176B parameters by BigScience Workshop?
- Lessons learned from training 104B model
t-zero
Posts with mentions or reviews of t-zero.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-03-15.
-
CarperAI announces plans for the first open-source “instruction-tuned” LM
I am very much wishing for the same. I have used T0_3B (https://github.com/bigscience-workshop/t-zero) on my M40 with 24GB of vRAM. It was fun to get it working, but its not remotely comparable to GPT-3 (instructGPT).
-
[Announcement] HuggingFace BigScience AMA Thursday, March 24th from 5pm CET
You have T0 that came from BigScience as well: https://github.com/bigscience-workshop/t-zero / https://huggingface.co/bigscience/T0
What are some alternatives?
When comparing bigscience and t-zero you can also consider the following projects:
lm-evaluation-harness - A framework for few-shot evaluation of language models.
enclosure-picroft - Mycroft interface for Raspberry Pi environment
cka-crash-course - In-depth and hands-on practice for acing the exam.
blackjack-discard-tray-photos - Sequential photos of cards piled in a discard tray (useful for deck estimation practice)
spacy-models - 💫 Models for the spaCy Natural Language Processing (NLP) library
Pytorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration
security-training - Public version of PagerDuty's employee security training courses.