Python ai-safety

Open-source Python projects categorized as ai-safety

Top 7 Python ai-safety Projects

  1. giskard

    🐢 Open-Source Evaluation & Testing for AI & LLM systems

    Project mention: Creation of the ApostropheCMS Documentation Chatbot | dev.to | 2024-08-29

    When originally designing the chatbot, we opted to build it in Python, despite being a heavily JavaScript-oriented shop. This decision was driven by the availability of more mature analytic tools for objectively testing chatbot hallucination and accuracy in Python. So far, we've been evaluating answers qualitatively, but we plan to incorporate a tool like Giskard to bring a more quantitative approach to our evaluations. This step is crucial and one that, anecdotally, is often overlooked in many production chatbots.

  2. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  3. safe-rlhf

    Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

  4. pegasi-shield

    Alignment toolkit to safeguards LLMs

  5. ethics

    Aligning AI With Shared Human Values (ICLR 2021)

  6. Thought-Cloning

    [NeurIPS '23 Spotlight] Thought Cloning: Learning to Think while Acting by Imitating Human Thinking

  7. ToolEmu

    [ICLR'24 Spotlight] A language model (LM)-based emulation framework for identifying the risks of LM agents with tool use

  8. moonwatcher

    Evaluation & testing framework for computer vision models

    Project mention: Open-Source Evaluation and Testing Framework for Computer Vision Models | news.ycombinator.com | 2024-06-03

    Hey,

    for the past weeks, we’ve been developing an open-source evaluation and testing framework for computer vision models. Today we’ve released the first alpha version and would love to get your feedback and support.

    Github: https://github.com/moonwatcher-ai/moonwatcher

    *What problems are we solving?*

    - *Manual, error-prone evaluation:* Assessing model quality is still a manual and error-prone process. Of course, aggregation metrics exist, but they usually overlook that the model works differently on some parts of the data.

  9. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python ai-safety discussion

Log in or Post with

Python ai-safety related posts

  • [R] Identifying the Risks of LM Agents with an LM-Emulated Sandbox - University of Toronto 2023 - Benchmark consisting of 36 high-stakes tools and 144 test cases!

    1 project | /r/MachineLearning | 11 Oct 2023
  • ToolEmu: Identifying the Risks of LM Agents with an LM-Emulated Sandbox

    1 project | news.ycombinator.com | 10 Oct 2023

Index

What are some of the best open-source ai-safety projects in Python? This list will help you:

# Project Stars
1 giskard 4,272
2 safe-rlhf 1,402
3 pegasi-shield 318
4 ethics 265
5 Thought-Cloning 260
6 ToolEmu 125
7 moonwatcher 16

Sponsored
CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai

Did you know that Python is
the 2nd most popular programming language
based on number of references?