Local-LLM-Langchain VS ragas

Which works, but produces nonsense 90% of the time for me. There's also that one notebook/colab for testing langchain but is above my abilities to use. And there's Auto-Vicuna of course, whose development however seems stalled

congrats on launching! i think my continuing struggle with looking at Ragas as a company rather than an oss library is that the core of it is like 8 metrics (https://github.com/explodinggradients/ragas/tree/main/src/ra...) that are each 1-200 LOC. i can inline that easily in my app and retain full control, or model that in langchain or haystack or whatever.

why is Ragas a library and a company, rather than an overall "standard" or philosophy (eg like Heroku's 12 Factor Apps) that could maybe be more robust?

(just giving an opp to pitch some underappreciated benefits of using this library)

Also, at some point you'll need to get serious about evaluation (trust me, you will). You may be interested in https://github.com/explodinggradients/ragas

The problem becomes worse if you want to inspect outputs from not just one, but several different queries. Luckily, there are several free open source packages such as ragas and DeepEval that can help evaluate your chatbot so you don't have to manually do it 😌

We have build RAGAS framework for this https://github.com/explodinggradients/ragas

Hi u/thecuteturtle, I am building open-source projects for evaluating LLM-based applications. Check it out https://github.com/explodinggradients/ragas and if you like to collaborate let me know :)