Python ai-safety

Open-source Python projects categorized as ai-safety
Edit details
Topics: large-language-models Artificial intelligence reinforcement-learning ml-safety ML
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 6 Python ai-safety Projects

  • giskard

    🐢 Open-Source Evaluation & Testing framework for LLMs and ML models

    • Project mention: Show HN: Evaluate LLM-based RAG Applications with automated test set generation | news.ycombinator.com | 2024-04-11

  • safe-rlhf

    Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

    • Project mention: [R] Meet Beaver-7B: a Constrained Value-Aligned LLM via Safe RLHF Technique | /r/MachineLearning | 2023-05-16

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • T-RAGS

    Trustworthy Retrieval Augmented Generation (RAG) with Safeguards

    • Project mention: Safeguard OpenAI Apps with Guardrail ML’s Firewall | /r/OpenAI | 2023-09-25

  • Thought-Cloning

    [NeurIPS '23 Spotlight] Thought Cloning: Learning to Think while Acting by Imitating Human Thinking

    • Project mention: AI Agents Can Learn to Think While Acting: A New AI Research Introduces A Novel Imitation Learning Framework Called Thought Cloning | /r/machinelearningnews | 2023-06-07

  • ethics

    Aligning AI With Shared Human Values (ICLR 2021)

  • ToolEmu

    A language model (LM)-based emulation framework for identifying the risks of LM agents with tool use

    • Project mention: [R] Identifying the Risks of LM Agents with an LM-Emulated Sandbox - University of Toronto 2023 - Benchmark consisting of 36 high-stakes tools and 144 test cases! | /r/MachineLearning | 2023-10-11

    Website: https://toolemu.com/

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python ai-safety related posts

  • [R] Identifying the Risks of LM Agents with an LM-Emulated Sandbox - University of Toronto 2023 - Benchmark consisting of 36 high-stakes tools and 144 test cases!

    1 project | /r/MachineLearning | 11 Oct 2023

  • ToolEmu: Identifying the Risks of LM Agents with an LM-Emulated Sandbox

    1 project | news.ycombinator.com | 10 Oct 2023

Index

What are some of the best open-source ai-safety projects in Python? This list will help you:

Project Stars
1 giskard 3,164
2 safe-rlhf 1,160
3 T-RAGS 311
4 Thought-Cloning 232
5 ethics 207
6 ToolEmu 86

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com