lm-evaluation-harness

A framework for few-shot evaluation of autoregressive language models. (by bigscience-workshop)

Lm-evaluation-harness Alternatives

Similar projects and alternatives to lm-evaluation-harness

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better lm-evaluation-harness alternative or higher similarity.

lm-evaluation-harness reviews and mentions

Posts with mentions or reviews of lm-evaluation-harness. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-04-19.
  • Stability AI Launches the First of Its StableLM Suite of Language Models
    24 projects | news.ycombinator.com | 19 Apr 2023
    Yeah, although looks like it currently has some issues with coqa: https://github.com/EleutherAI/lm-evaluation-harness/issues/2...

    There's also the bigscience fork, but I ran into even more problems (although I didn't try too hard) https://github.com/bigscience-workshop/lm-evaluation-harness

    And there's https://github.com/EleutherAI/lm-eval2/ (not sure if it's just starting over w/ a new repo or what?) but it has limited tests available

Stats

Basic lm-evaluation-harness repo stats
1
91
3.7
12 months ago

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com