ChatGPT_for_IE

Evaluating ChatGPT’s Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness (by pkuserc)

ChatGPT_for_IE Alternatives

Similar projects and alternatives to ChatGPT_for_IE based on common topics and language

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better ChatGPT_for_IE alternative or higher similarity.

ChatGPT_for_IE reviews and mentions

Posts with mentions or reviews of ChatGPT_for_IE. We have used some of these posts to build our list of alternatives and similar projects.
  • Evaluating ChatGPT's Information Extraction Capabilities: An Assessment
    1 project | news.ycombinator.com | 25 Apr 2023
    Abstract: The capability of Large Language Models (LLMs) like ChatGPT to comprehend user intent and provide reasonable responses has made them extremely popular lately. In this paper, we focus on assessing the overall ability of ChatGPT using 7 fine-grained information extraction (IE) tasks. Specially, we present the systematically analysis by measuring ChatGPT's performance, explainability, calibration, and faithfulness, and resulting in 15 keys from either the ChatGPT or domain experts. Our findings reveal that ChatGPT's performance in Standard-IE setting is poor, but it surprisingly exhibits excellent performance in the OpenIE setting, as evidenced by human evaluation. In addition, our research indicates that ChatGPT provides high-quality and trustworthy explanations for its decisions. However, there is an issue of ChatGPT being overconfident in its predictions, which resulting in low calibration. Furthermore, ChatGPT demonstrates a high level of faithfulness to the original text in the majority of cases. We manually annotate and release the test sets of 7 fine-grained IE tasks contains 14 datasets to further promote the research. The datasets and code are available at this https URL - https://github.com/pkuserc/ChatGPT_for_IE

Stats

Basic ChatGPT_for_IE repo stats
1
133
8.1
6 months ago

pkuserc/ChatGPT_for_IE is an open source project licensed under Apache License 2.0 which is an OSI approved license.

The primary programming language of ChatGPT_for_IE is Python.


Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com