chain-of-thought-hub
datablations
| chain-of-thought-hub | datablations | |
|---|---|---|
| 10 | 6 | |
| 2,371 | 289 | |
| - | 3.5% | |
| 6.9 | 6.9 | |
| 10 days ago | about 1 month ago | |
| Jupyter Notebook | Jupyter Notebook | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
chain-of-thought-hub
- Chain-Of-Thought Hub: Measuring LLMs' Reasoning Performance
-
All Model Leaderboards (that I know)
Chain-of-Thought Hub https://github.com/FranxYao/chain-of-thought-hub - these are mostly gathered although Yao Fu, the author is working on specific CoT runs
- It looks likely that the MMLU score on Hugginface's LLM leaderboard is wrong after all.
-
(2/2) May 2023
Chain-of-Thought Hub: Measuring LLMs' Reasoning Performance (https://github.com/FranxYao/chain-of-thought-hub)
-
Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
https://github.com/FranxYao/chain-of-thought-hub
- [N] Chain-of-Thought Hub: Measuring LLMs' Reasoning Performance
- Chain-of-Thought Hub: Measuring LLMs' Reasoning Performance
datablations
-
Gemini is only 1x Chinchilla, so it undertrained for production
1x chinchilla means it's not really undertrained but that more could be squeezed without excessive difficulty https://arxiv.org/abs/2305.16264
- Can LLMs learn from a single example?
-
Chinchilla’s Death
You might want to give a read to "Scaling Data-Constrained Language Models" [1]. They basically generalized the Chinchilla scaling law by investigating behavior on multi-epoch runs.
[1] https://arxiv.org/abs/2305.16264
-
RWKV Pile+ seems to be training on far more tokens than any LLM ever has
I would imagine that there is a lot of overlap, yeah. That said, training on repeated data does seem to be effective at this level.
-
(2/2) May 2023
Scaling Data-Constrained Language Models (https://arxiv.org/abs/2305.16264)
- How to Keep Scaling Large Language Models when Data Runs Out? A New AI Research Trains 400 Models with up to 9B Parameters and 900B Tokens to Create an Extension of Chinchilla Scaling Laws for Repeated Data
What are some alternatives?
DB-GPT - AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents
TinyLlama - The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
llm-leaderboard - A joint community effort to create one central leaderboard for LLMs.
airoboros - Customizable implementation of the self-instruct paper.
tree-of-thoughts - Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%
prompt-engineering - Tips and tricks for working with Large Language Models like OpenAI's GPT-4.
llm-humaneval-benchmarks
SuperAGI - <⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
GirlfriendGPT - Girlfriend GPT is a Python project to build your own AI girlfriend using ChatGPT4.0
chathub - All-in-one chatbot client