[R] Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet

• do-you-even-need-attention

  2 478 0.0 Python

  Is the attention layer even necessary? (https://arxiv.org/abs/2105.02723)

For those who are interested, here's a GitHub repo with pretrained models, a W&B log of the experiments, and a 3-page writeup.

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

Suggest a related project