DeepRobust
auto-attack
Our great sponsors
DeepRobust | auto-attack | |
---|---|---|
1 | 3 | |
940 | 607 | |
- | - | |
5.5 | 0.0 | |
5 days ago | 3 months ago | |
Python | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
DeepRobust
auto-attack
-
DARPA Open Sources Resources to Aid Evaluation of Adversarial AI Defenses
I'm less familiar with poisoning, but at least for test-time robustness, the current benchmark for image classifiers is AutoAttack [0,1]. It's an ensemble of adaptive & parameter-free gradient-based and black-box attacks. Submitted academic work is typically considered incomplete without an evaluation on AA (and sometimes deepfool [2]). It is good to see that both are included in ART.
[0] https://arxiv.org/abs/2003.01690
[1] https://github.com/fra31/auto-attack
[2] https://arxiv.org/abs/1511.04599
-
[D] Testing a model's robustness to adversarial attacks
A better method is to use the AutoAttack from Croce et al. https://github.com/fra31/auto-attack which is much more robust to gradient masking. It's actually a combination of 3 attacks (2 white-box and 1 black box) with good default hyper-parameters. It's not perfect but it gives a more accurate robustness.
What are some alternatives?
adversarial-robustness-toolbox - Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
torchdrug - A powerful and flexible machine learning platform for drug discovery
TextAttack - TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/
ccat - Cisco Config Analysis Tool
KitanaQA - KitanaQA: Adversarial training and data augmentation for neural question-answering models
text_gcn - Graph Convolutional Networks for Text Classification. AAAI 2019
alpha-beta-CROWN - alpha-beta-CROWN: An Efficient, Scalable and GPU Accelerated Neural Network Verifier (winner of VNN-COMP 2021, 2022, and 2023)