long-form-factuality
Benchmarking long-form factuality in large language models. Original code for our paper "Long-form factuality in large language models". (by google-deepmind)
FActScore
A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation" (by shmsw25)
| long-form-factuality | FActScore | |
|---|---|---|
| 2 | 1 | |
| 447 | 217 | |
| 80.5% | - | |
| 6.3 | 6.4 | |
| 6 days ago | 3 months ago | |
| Python | Python | |
| GNU General Public License v3.0 or later | MIT License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
long-form-factuality
Posts with mentions or reviews of long-form-factuality.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2024-04-06.
-
An Open Source Tool for Multimodal Fact Verification
Isn't this similar to the Deepmind paper on long form factuality posted a few days ago?
https://arxiv.org/abs/2403.18802
https://github.com/google-deepmind/long-form-factuality/tree...
- LongFact – Long-Form Factuality in Large Language Models
FActScore
Posts with mentions or reviews of FActScore.
We have used some of these posts to build our list of alternatives
and similar projects.
-
Long-form factuality in large language models
Looks like a slight modification of FActScore [1], but instead of using Wikipedia as a verification source, they use Google Search. They also claim to include a wider range of topics. That said, FActScore allows you to use whatever knowledge source and topics you want [2].
[1]: https://arxiv.org/abs/2305.14251
[2]: https://github.com/shmsw25/FActScore?tab=readme-ov-file#to-u...