Dataset of MMLU results broken down by task

• lm-evaluation-harness

  34 5,070 9.9 Python

  A framework for few-shot evaluation of language models.

  

I am primarily looking for results of running the MMLU evaluation on modern large language models. I have been able to find some data here https://github.com/EleutherAI/lm-evaluation-harness/tree/master/results and will be asking them if/when, they can provide any additional data.

• helm

  2 1,654 9.8 Python

  Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in Holistic Evaluation of Text-to-Image Models (HEIM) (https://arxiv.org/abs/2311.04287). (by stanford-crfm)

  

Looking at their github repo, it also seems like the MMLU result is from just those 5 tasks and not all of them https://github.com/stanford-crfm/helm/issues/1335

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project