-
alpaca_eval
An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
-
alpaca_farm
A simulation framework for RLHF and alternatives. Develop your RLHF method without collecting human data.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
an automatic evaluator that is easy to use, fast, cheap and validated against 20K human annotations. It actually has a higher agreement with majority vote of humans than a single human annotator! Of course, our method still has limitations which we discuss here!
AlpacaEval dataset: 805 instructions, which are a simplification of AlpacaFarm's evaluation set.