deep-vector-quantization VS chainer-VQ-VAE

How the codebook is initialised makes a big difference, if more codes are used at the start then they often stay in use but it can be finicky to figure that out. I've found initialising codes normally with small standard deviation (e.g. 0.01) to help (so codes lie on a hypersphere). Failing that, here is an example with k-means initialisation https://github.com/karpathy/deep-vector-quantization .

The context is, that I am trying to learn a discrete vocabulary of latent codes, i.e. have a discrete learnable embedding to sort of quantize the otherwise continuous latent outputs of the encoder that are then used to reconstruct the input image via the decoder cf. this code snippet. So the idea is not to generate random sampled from noise but to learn an efficient notebook, i.e. bottleneck that captures the essentials of the data set. The decoder then outputs a prob distribution for every pixel over the 255 possible values 8 bit images can take o. The KL (assuming a uniform prior to encourage uniform use of all possible vocabulary entries) is currently weighted with 1.

Code for https://arxiv.org/abs/1711.00937 found: https://github.com/dhgrs/chainer-VQ-VAE