Is anyone else super eager to upgrade their computer but they're also trying to patiently wait to see what might come out? What's your game plan?

• llama.cpp

  775 57,463 10.0 C++

  LLM inference in C/C++

I haven't benchmarked it yet, I always use -ins or -r so I don't see the tokens/s. I plan to check that tonight and I can share the results. I started running the version with dfyz's improvements to avx-512 performance https://github.com/ggerganov/llama.cpp/pull/933 recently. It's very slow, sure, but I like the output superiority of 65b so much that I don't want to settle for 33B even though it's 2 times faster and not that much dumber.

• BIG-bench

  45 2,660 0.0 Python

  Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

  

After that, if I'm not dead yet, I want to go after it with BIG-bench. I almost made a feature request of Oobabooga, but I don't think many other people have this dream.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• lm-evaluation-harness

  34 5,070 9.9 Python

  A framework for few-shot evaluation of language models.

  

BTW, if you're looking to do benchmarking against other setupts, I'd recommend using lm-eval instead. You have a lot of benchmarkts to pick from and can easily compare against what other people have run. Fabrice Bellard (yeah, that one) has benched a huge number of open LLMs, including at different sizes and quantizations: https://bellard.org/ts_server/

• koboldcpp

  180 3,817 10.0 C++

  A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

Suggest a related project