Our great sponsors
-
parti-pytorch
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
-
Swin-Transformer
This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Give it a few days and lucidrains will have the code up[0].
But in honesty, it is probably how people react. We saw this with Pulse, GPT, and many others. The authors are clear about the limitations but people talk it up too much and others shit on it. There's also a reproducibility crisis in ML (many famous networks, like Swin[1][2][3], can't be reproduced (even worse when reviewers concentrate on benchmarks)). It isn't like many can train a model like this anyways. It gives them benefit of the doubt and maintains good publicity rather than controversial.
Of course, this is extremely bad from an academic perspective and personally I believe you should have your paper revoked if it isn't reproducible. You'd be surprised how many don't track the random seed or measure variance. We have GitHub. You should be able to write training options that get approximately the same results as the paper. Otherwise I don't trust your results.
[0] https://github.com/lucidrains/parti-pytorch
[1] https://github.com/microsoft/Swin-Transformer/issues/183
[2] https://github.com/microsoft/Swin-Transformer/issues/180
[3] https://github.com/microsoft/Swin-Transformer/issues/148
Give it a few days and lucidrains will have the code up[0].
But in honesty, it is probably how people react. We saw this with Pulse, GPT, and many others. The authors are clear about the limitations but people talk it up too much and others shit on it. There's also a reproducibility crisis in ML (many famous networks, like Swin[1][2][3], can't be reproduced (even worse when reviewers concentrate on benchmarks)). It isn't like many can train a model like this anyways. It gives them benefit of the doubt and maintains good publicity rather than controversial.
Of course, this is extremely bad from an academic perspective and personally I believe you should have your paper revoked if it isn't reproducible. You'd be surprised how many don't track the random seed or measure variance. We have GitHub. You should be able to write training options that get approximately the same results as the paper. Otherwise I don't trust your results.
[0] https://github.com/lucidrains/parti-pytorch
[1] https://github.com/microsoft/Swin-Transformer/issues/183
[2] https://github.com/microsoft/Swin-Transformer/issues/180
[3] https://github.com/microsoft/Swin-Transformer/issues/148
Related posts
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
- Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows
- [D] Influential papers round-up 2022. What are your favorites?
- Using VIT as a feature extractor
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows