Top 6 Python ml-safety Projects
-
When originally designing the chatbot, we opted to build it in Python, despite being a heavily JavaScript-oriented shop. This decision was driven by the availability of more mature analytic tools for objectively testing chatbot hallucination and accuracy in Python. So far, we've been evaluating answers qualitatively, but we plan to incorporate a tool like Giskard to bring a more quantitative approach to our evaluations. This step is crucial and one that, anecdotally, is often overlooked in many production chatbots.
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
-
-
-
-
ModelNet40-C
Repo for "Benchmarking Robustness of 3D Point Cloud Recognition against Common Corruptions" https://arxiv.org/abs/2201.12296
-
Project mention: Open-Source Evaluation and Testing Framework for Computer Vision Models | news.ycombinator.com | 2024-06-03
Hey,
for the past weeks, we’ve been developing an open-source evaluation and testing framework for computer vision models. Today we’ve released the first alpha version and would love to get your feedback and support.
Github: https://github.com/moonwatcher-ai/moonwatcher
*What problems are we solving?*
- *Manual, error-prone evaluation:* Assessing model quality is still a manual and error-prone process. Of course, aggregation metrics exist, but they usually overlook that the model works differently on some parts of the data.
Python ml-safety discussion
Python ml-safety related posts
Index
What are some of the best open-source ml-safety projects in Python? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | giskard | 4,359 |
| 2 | natural-adv-examples | 601 |
| 3 | langtest | 516 |
| 4 | ethics | 265 |
| 5 | ModelNet40-C | 203 |
| 6 | moonwatcher | 16 |
Sponsored