cleverhans VS WeightWatcher

I only knew of this story from looking up the name of this library on adversarial DL https://github.com/cleverhans-lab/cleverhans

I feel as though this area has not received much attention over the last couple of years. The CleverHans project has gone stale and I haven't heard of many new results recently. Has the community lost interest in this area? Did we decide that adversarial attacks aren't such a problem in practical applications?

These models are pretty much always extrapolating [0]

Whether the extrapolation is crude/low-rank or astute/high-rank is a question of memorization vs generalization. That gets into the question of whether or not the model is over-fitted or under-fitted. There are certain heuristics borrowed from high dimensional statistical physics that can be used to guess how good the test performance of a model will be on a typical task without even knowing what the test data is [1].

Originality for me means finding better answers to sub-tasks, and then combining those answers together in a better way. This is the nirvana of cross-entropy minimization - the emergence of capability results from gaining the ability to amass a wider range of skills, improving upon them, and percolating those improvements towards multiply the leverage of other skills.

How long such a thing can keep improving with current tech, who knows, but you should really think critically about whether that sounds just like interpolation through the corpus.

[0] Learning in High Dimension Always Amounts to Extrapolation - https://arxiv.org/abs/2110.09485

[1] https://github.com/CalculatedContent/WeightWatcher

One of the things I love about physics is that, in addition to probably being my favorite of study in it's own right, it seems that a lot of the conceptual/mathematical content carries over and contributes to other fields. One example I've come across recently can be found here: https://github.com/CalculatedContent/WeightWatcher and here: