OpenFactCheck Documentation#
OpenFactCheck is an open source, Apache-licensed library designed to facilitate the evaluation and enhancement of factuality in responses generated by large language models (LLMs).
The design of openfactcheck emphasizes two principles:
Customizability and extensibility for both users and developers
Compatibility with existing methods and datasets
It consists of three core modules:
Response Evaluator
Customizes an automatic fact-checker to verify free-form documents, providing detailed reports on the factuality of the responses.
LLM Evaluator
Applies factuality-specific benchmarks to assess the factual performance of large language models, providing detailed reports on the factuality of the responses.
Fact Checker Evaluator
Evaluates fact-checking accuracy, featuring a leaderboard that ranks systems by precision, speed, and cost, driving innovation in automatic fact-checking.
Explore the documentation to learn more about the library, its features, how to use and contribute to it, and more.
Warning
The API and documentation are still under heavy development and may change in future versions.
New to OpenFactCheck? Check out the getting started guides. They contain an introduction to OpenFactCheck’s main concepts and links to additional tutorials.
The user guide provides in-depth information on the key concepts of OpenFactCheck with useful background information and explanation.
The reference guide contains a detailed description of the OpenFactCheck API. The reference describes how the methods work and which parameters can be used. It assumes that you have an understanding of the key concepts.
Want to improve existing functionalities? Saw a typo in the documentation? Want to contribute to the project? The contributing guidelines will guide you through the process of improving OpenFactCheck.
Useful links: Binary Installers | Source Repository | Issues & Ideas |