openplayground VS promptfoo

https://github.com/nat/openplayground

I load up $5 into my account using my credit card and then reload it whenever it gets low, it also has a tab for comparing multiple resulta from different models together.

https://github.com/nat/openplayground https://discord.gg/uT98U9HJ.

Try here: https://github.com/nat/openplayground

From there you have lots of other models: One of the best places to easily start using multiple models is using a multiple model UI program lik GPT4All, there are also some programs that provide access to more models or use different ways of interfacing with them, here are some of what I've found are the best / most popular programs to play around with lots of different models and compare them: LocalAI, text-generation-webui, open playground

Here's another interesting engine called AI playground that lets you do side-by-side comparisons of language models based on the same prompts: https://github.com/nat/openplayground

Super interesting. We've been experimenting with [promptfoo](https://github.com/promptfoo/promptfoo) at my work, and this looks very similar.

In case anyone's interested in running their own benchmark across many LLMs, I've built a generic harness for this at https://github.com/promptfoo/promptfoo.

I encourage people considering LLM applications to test the models on their _own data and examples_ rather than extrapolating general benchmarks.

This library supports OpenAI, Anthropic, Google, Llama and Codellama, any model on Replicate, and any model on Ollama, etc. out of the box. As an example, I wrote up an example benchmark comparing GPT model censorship with Llama models here: https://promptfoo.dev/docs/guides/llama2-uncensored-benchmar.... Hope this helps someone.