Adan
AdamP
Adan | AdamP | |
---|---|---|
1 | 1 | |
727 | 409 | |
1.2% | 1.5% | |
4.6 | 0.0 | |
about 1 month ago | over 3 years ago | |
Python | Python | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Adan
-
Find Optimal Learning Rates for Stable Diffusion Fine-tunes (Link in Comments)
Adan https://github.com/sail-sg/Adan
AdamP
-
[R] A theoretical reviewing video on "AdamP: Slowing Down the Slowdown for momentum optimizers (ICLR 2021)"
Paper: https://openreview.net/forum?id=Iz3zU3M316D Code: https://github.com/clovaai/AdamP Project page: https://clovaai.github.io/AdamP/
What are some alternatives?
Sophia - Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.
horovod - Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
Adan-pytorch - Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch
sam - SAM: Sharpness-Aware Minimization (PyTorch)
dadaptation - D-Adaptation for SGD, Adam and AdaGrad
OASIS - Official implementation of the paper "You Only Need Adversarial Supervision for Semantic Image Synthesis" (ICLR 2021)
pytorch_resnet_cifar10 - Proper implementation of ResNet-s for CIFAR10/100 in pytorch that matches description of the original paper.
EasyOCR - Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.