neural-tangents VS eigenlearning

is used to capture the power of a fully-trained deep net of infinite width.

https://openreview.net/pdf?id=rkl4aESeUH, https://github.com/google/neural-tangents

> It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width.

https://arxiv.org/abs/1711.00165

And of course, one needs to look back at SVMs applying a kernel function and separating with a line, which looks a lot like an ANN with a single hidden layer followed by a linear mapping.

https://stats.stackexchange.com/questions/238635/kernel-meth...

Code for https://arxiv.org/abs/2011.00050 found: https://github.com/google/neural-tangents

I saw that you asked about neural tangent kernels (NTK) in another post yesterday -- be aware that what you're referencing in the present post are "neural network gaussian processes" (NNGP), which is distinct from NTK! The README of https://github.com/google/neural-tangents should help lift confusion. (I also took the term NNGP from there.)

It's true! There have been dozens of papers published on this topic, some of which are listed here: https://github.com/google/neural-tangents#references

In addition to using a validation set, there is research w.r.t. the neural tangent kernel which claims that metrics correlated to both training speed and generalization can be computed from the NTK. This would then remove/reduce the need for training as one can substitute it with computing the NTK. I don’t have a complete list of references, but here is one example and another where they have applied NTK (and number of linear regions) to NAS.

Code for https://arxiv.org/abs/2110.03922 found: https://github.com/james-simon/eigenlearning