Top 7 Python ai-safety Projects
-
When originally designing the chatbot, we opted to build it in Python, despite being a heavily JavaScript-oriented shop. This decision was driven by the availability of more mature analytic tools for objectively testing chatbot hallucination and accuracy in Python. So far, we've been evaluating answers qualitatively, but we plan to incorporate a tool like Giskard to bring a more quantitative approach to our evaluations. This step is crucial and one that, anecdotally, is often overlooked in many production chatbots.
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
-
safe-rlhf
Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback
-
-
-
Thought-Cloning
[NeurIPS '23 Spotlight] Thought Cloning: Learning to Think while Acting by Imitating Human Thinking
-
ToolEmu
[ICLR'24 Spotlight] A language model (LM)-based emulation framework for identifying the risks of LM agents with tool use
-
Project mention: Open-Source Evaluation and Testing Framework for Computer Vision Models | news.ycombinator.com | 2024-06-03
Hey,
for the past weeks, we’ve been developing an open-source evaluation and testing framework for computer vision models. Today we’ve released the first alpha version and would love to get your feedback and support.
Github: https://github.com/moonwatcher-ai/moonwatcher
*What problems are we solving?*
- *Manual, error-prone evaluation:* Assessing model quality is still a manual and error-prone process. Of course, aggregation metrics exist, but they usually overlook that the model works differently on some parts of the data.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Python ai-safety discussion
Python ai-safety related posts
Index
What are some of the best open-source ai-safety projects in Python? This list will help you:
# | Project | Stars |
---|---|---|
1 | giskard | 4,272 |
2 | safe-rlhf | 1,402 |
3 | pegasi-shield | 318 |
4 | ethics | 265 |
5 | Thought-Cloning | 260 |
6 | ToolEmu | 125 |
7 | moonwatcher | 16 |