HALOs

A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs). (by ContextualAI)
Add to my DEV experience Alignment dpo halos kto Ppo rlhf
Source Code
arxiv.org
Suggest alternative
Edit details
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

HALOs Alternatives

Similar projects and alternatives to HALOs based on common topics and language

  • argilla

    15 HALOs VS argilla

    Argilla is a collaboration platform for AI engineers and domain experts that require high-quality outputs, full data ownership, and overall efficiency.

  • LLMSurvey

    3 HALOs VS LLMSurvey

    The official GitHub page for the survey paper "A Survey of Large Language Models".

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • LLaMA-Factory

    2 HALOs VS LLaMA-Factory

    Unify Efficient Fine-Tuning of 100+ LLMs

  • Open-Assistant

    329 HALOs VS Open-Assistant

    OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

  • alignment-handbook

    3 HALOs VS alignment-handbook

    Robust recipes to align language models with human and AI preferences

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better HALOs alternative or higher similarity.
Suggest an alternative to HALOs

HALOs reviews and mentions

Posts with mentions or reviews of HALOs. We have used some of these posts to build our list of alternatives and similar projects.
  • On Sleeper Agent LLMs
    1 project | news.ycombinator.com | 13 Jan 2024
    If you are using no-code solutions, increasing an "idea" in a dataset will make that idea more likely to appear.

    If you are fine-tuning your own LLM, there are other ways to get your idea to appear. In the literature this is sometimes called RLHF or preference optimization, and here are a few approaches:

    Direct Preference Optimization

    This uses Elo-scores to learn pairwise preferences. Elo is used in chess and basketball to rank individuals who compete in pairs.

    @argilla_io on X.com has been doing some work in evaluating DPO.

    Here is a decent thread on this: https://x.com/argilla_io/status/1745057571696693689?s=20

    Identity Preference Optimization

    IPO is research from Google DeepMind. It removes the reliance of Elo scores to address overfitting issues in DPO.

    Paper: https://x.com/kylemarieb/status/1728281581306233036?s=20

    Kahneman-Tversky Optimization

    KTO is an approach that uses mono preference data. For example, it asks if a response is "good or not." This is helpful for a lot of real word situations (e.g. "Is the restaurant well liked?").

    Here is a brief discussion on it:

    https://x.com/ralphbrooks/status/1744840033872330938?s=20

    Here is more on KTO:

    * Paper: https://github.com/ContextualAI/HALOs/blob/main/assets/repor...

    * Code: https://github.com/ContextualAI/HALOs

Stats

Basic HALOs repo stats
1
541
8.8
4 days ago

ContextualAI/HALOs is an open source project licensed under Apache License 2.0 which is an OSI approved license.

The primary programming language of HALOs is Python.

Popular Comparisons

  1. HALOs VS argilla
  2. HALOs VS LLMSurvey
  3. HALOs VS LLaMA-Factory

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com