BooookScore
A package to generate summaries of long-form text and evaluate the coherence of these summaries. Official package for our ICLR 2024 paper, "BooookScore: A systematic exploration of book-length summarization in the era of LLMs". (by lilakk)
ragrank
🎯 Your free LLM evaluation toolkit helps you assess the accuracy of facts, how well it understands context, its tone, and more. This helps you see how good your LLM applications are. (by Auto-Playground)
| BooookScore | ragrank | |
|---|---|---|
| 1 | 1 | |
| 75 | 21 | |
| - | - | |
| 7.0 | 9.5 | |
| 2 months ago | 12 days ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
BooookScore
Posts with mentions or reviews of BooookScore.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2024-04-09.
-
Evaluating faithfulness and content selection of LLMs in book-length summaries
With a link to https://arxiv.org/pdf/2310.00785.pdf - which then links to another GitHub repository, https://github.com/lilakk/BooookScore which has a bunch of prompts in https://github.com/lilakk/BooookScore/tree/main/prompts
Which makes me think that this original paper isn't evaluating LLMs so much as it's evaluating that one particular prompting technique for long summaries.
Gemini Pro 1.5 has 1m token context length, which should remove the need for weird hierarchical summary tricks. I wonder how well it would score?
ragrank
Posts with mentions or reviews of ragrank.
We have used some of these posts to build our list of alternatives
and similar projects.
-
I created Ragrank 🎯- An open source ecosystem to evaluate LLM and RAG.
Feel free to contribute on GitHub 💚