ddsp VS Kornia

You can actually use thousands of oscillators, which is what Magenta did with their differentiable DSP (DDSP) approach to modeling acoustic instruments including singing:

https://magenta.tensorflow.org/ddsp-vst

https://magenta.tensorflow.org/ddsp

I doesn't sound realistic but it is expressive in an unusual way.

Have you checked out DDSP from magenta? I’m not sure what kind of learning you are trying to implement but they provide a lot of standard signal processing components that are differentiable so you can use them as part of a neural net. There are a handful of collabs that serve as interactive tutorials and the documentation is not bad, I did my bachelors thesis on the topic so I got quite stuck into this for a while

In case you're asking about the parent comment (s/Differential/Differentiable), it means the title should be "differentiable CV" (instead of "differential CV").

"Differentiable" describes some computation for which derivatives can be computed. "Differential" is a more general term which means that something has to do with differences, e.g. differential equations deal with equations that specify how things change together.

With the recent surge in deep learning came significant improvements in optimization techniques and hardware, making it feasible to formulate some computations in a differentiable manner. Doing that allows one to optimize the computation process relatively efficiently, at least in theory. Some other examples: differentiable programming[0] (other differentiable techniques are a subset of this), differentiable rendering[1], differentiable signal processing[2].

[0] https://en.wikipedia.org/wiki/Differentiable_programming

[1] https://arxiv.org/abs/2006.12057

[2] https://github.com/magenta/ddsp

👉 https://github.com/kornia/kornia/releases/tag/v0.6.6

Kornia: Differential Computer Vision\ (3 comments)

Kornia - Kornia is a differentiable computer vision library for PyTorch. It consists of a set of routines and differentiable modules to solve generic computer vision problems. At its core, the package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions.

Between "not being sure yet" about GPU operations in pre-processing and choosing high-end CPUs, I think you are overthinking the wrong alternative. Besides DALI, check whether you are using codecs besides nvidia/torchvision-supported jpeg and png, and if other GPU CV libraries meet your needs: torchvision kornia

Use kornia.augmentation where this problem is solved doing the augmentations in batch outside the dataloader. https://github.com/kornia/kornia