pytorch-fid VS mugl

The Fréchet Inception Distance (FID) is a widespread metric to assess the quality of the distribution of a image generative model (GAN, Stable Diffusion, etc.). The metric is not trivial to implement as one needs to compute the trace of the square root of a matrix. In all PyTorch repositories I have seen that implement the FID (https://github.com/mseitzer/pytorch-fid, https://github.com/GaParmar/clean-fid, https://github.com/toshas/torch-fidelity, ...), the authors rely on SciPy's sqrtm to compute the square root of the matrix, which is unstable and slow.

https://github.com/GaParmar/clean-fid/ is my goto. https://github.com/mseitzer/pytorch-fid isn't bad either.

https://github.com/mseitzer/pytorch-fid for example this here. The code is quite clean and clear

Didn't get your problem statement but you can also consider FFID score to measure the similarity between two images. Check out: https://github.com/mseitzer/pytorch-fid

Author here, thanks for the comment! The example in 3.1 is not meant as a realistic example, it is meant as an intuitive explanation of how the manifold overfitting issue can arise. As for FID scores, we intend on releasing our code soon, but we used the standard PyTorch implementation (https://github.com/mseitzer/pytorch-fid).

In our work on generating human action sequences (https://github.com/skelemoa/mugl), we found FID to be poorly correlated with generation quality. But the community persists with the measure for some unknown reason. We found variants of Minimum Mean Discrepancy to be much better. This is for sequential data, though.