BooookScore

A package to generate summaries of long-form text and evaluate the coherence of these summaries. Official package for our ICLR 2024 paper, "BooookScore: A systematic exploration of book-length summarization in the era of LLMs". (by lilakk)
Add to my DEV experience Evaluation iclr2024 language-model llm NLP Summarization
Source Code
arxiv.org
Suggest alternative
Edit details
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

BooookScore Alternatives

Similar projects and alternatives to BooookScore based on common topics and language

  • FABLES

    1 BooookScore VS FABLES

  • haystack

    55 BooookScore VS haystack

    :mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • ragrank

    1 BooookScore VS ragrank

    🎯 Your free LLM evaluation toolkit helps you assess the accuracy of facts, how well it understands context, its tone, and more. This helps you see how good your LLM applications are.

  • txtai

    356 BooookScore VS txtai

    💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

  • LMOps

    6 BooookScore VS LMOps

    General technology for enabling AI capabilities w/ LLMs and MLLMs

  • xgen

    2 BooookScore VS xgen

    Salesforce open-source LLMs with 8k sequence length.

  • searchGPT

    3 BooookScore VS searchGPT

    Grounded search engine (i.e. with source reference) based on LLM / ChatGPT / OpenAI API. It supports web search, file content search etc.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better BooookScore alternative or higher similarity.
Suggest an alternative to BooookScore

BooookScore reviews and mentions

Posts with mentions or reviews of BooookScore. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2024-04-09.
  • Evaluating faithfulness and content selection of LLMs in book-length summaries
    2 projects | news.ycombinator.com | 9 Apr 2024
    With a link to https://arxiv.org/pdf/2310.00785.pdf - which then links to another GitHub repository, https://github.com/lilakk/BooookScore which has a bunch of prompts in https://github.com/lilakk/BooookScore/tree/main/prompts

    Which makes me think that this original paper isn't evaluating LLMs so much as it's evaluating that one particular prompting technique for long summaries.

    Gemini Pro 1.5 has 1m token context length, which should remove the need for weird hierarchical summary tricks. I wonder how well it would score?

Stats

Basic BooookScore repo stats
1
69
7.0
about 1 month ago

lilakk/BooookScore is an open source project licensed under MIT License which is an OSI approved license.

The primary programming language of BooookScore is Python.


Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com