Our great sponsors
-
SGD-OGR-Hessian-estimator
SGD (stochastic gradient descent) with OGR - online gradient regression Hessian estimator
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
So it directly estimates learning rates based on local trend of gradients, here is analogous scenario for standard momentum methods with maximal fixed learning rate: much smaller steps, after 30 steps ~50x worse values: https://github.com/JarekDuda/SGD-OGR-Hessian-estimator/raw/main/momentum.png
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a more popular project.
Related posts
- AArch64-Explore/vol1 M1 Explainer [pdf]
- Set 10 Tome of Traits Guide
- Mathematica notebook to check the math of "The Hydrino Hypothesis" at substack's "Hydrogen Revolution"
- Correction To My Mathematica GUTCP Cosmology Notebook: 120 Billion Years Since Start of the Last Expansion Phase
- I've done it, I beat Space Exploration's (v0.5) Secret Ending in 673 hours!