OpenNMT-py VS espnet

I already tried this with OpenNMT.

fairseq and OpenNMT are very good starting points if you want to train your NMT model from scratch.

It looks like this uses the version of OpenNMT implemented in torch, which has been deprecated. You will be much better off using the pytorch implementation of OpenNMT or the transformers library. In fact, I would recommend taking a look at the GEM benchmark, since it also uses the WebNLG dataset. Here is a tutorial to get started, you can change the dataset here to WebNLG instead of CommonGen.

Umm... open-nmt This is a library maintained since 2016 for NMT

However, you do not need to use much OOP when training models with pytorch. Most of the time it is just inheriting a class and overwriting functions. You might need more advanced stuff if you were writing a framework on top of it, something like ONMT

You might check out this list from espnet. They list the different corpuses they use to train their models sorted by language and task (ASR, TTS etc):

https://github.com/espnet/espnet/blob/master/egs2/README.md

- https://github.com/espnet/espnet

This link has instructions for doing this for a Japanese model. It would have to be altered to work with ljspeech and the fine tune dataset.

This work is made possible by the excellent advancements in text to speech modeling. ESPnet is a great project and should be checked out for more advanced and a wider range of use cases. This pipeline was also made possible by the great work from espnet_onnx in building a framework to export models to ONNX.

CMU WavLab has ESPNet https://espnet.github.io/espnet/ which includes a number of high quality TTS models including VITS (which in my subjective experience is just as good as what is demonstrated here). Also the inference on various ESPNet pretrained TTS models is reasonable and sentences take on average 5 seconds per word to generate the waveform on my totally mid PC setup.

The reason I'm saying this is to point out that having and in-depth knowledge on speech processing/generation requires a lot of information about signal processing and human speech in general (eg. acoustics and phonetics). However, if you're not into learning everything there is to know about a subject, just take one state-of-the-art example and study that as best as you can. Pick one environment/toolkit, for example espnet and simply go with that.

https://github.com/espnet/espnet (kind of like a newer Kaldi, but also not beginner friendly)

both provide e2e ASR support but espnet does have more utilities where as speechbarain is clean

This is relatively small amount of speech to train the model from scratch, but you can train using another pre-trained model for initialization. There are numbers of end-to-end ASR toolkits which can be used for this: https://github.com/NVIDIA/NeMo and https://github.com/espnet/espnet