Pink-Trombone VS tacotron2

Too late to edit, but to any one who needs "convincing" of the flexibility of a formant synthesizer, you should 1) play with Pink Trombone[1], a Javascript formant synthesizer with a UI that lets you graphically manipulate a vocal tract, and 2) have a look at this programmable version of it[2]

[1] https://dood.al/pinktrombone/

[2] https://github.com/zakaton/Pink-Trombone

Can I do it automatically without any audio/voice recordings at all, such as taking advantage of the pink trombone linguistics tool, (for which there is some source code)?

The quality also depends on the type of model. I'm not really sure what ESpeak-ng actually uses? The classical TTS approaches often use some statistical model (e.g. HMM) + some vocoder. You can get to intelligible speech pretty easily but the quality is bad (w.r.t. how natural it sounds).

There are better open source TTS models. E.g. check https://github.com/neonbjb/tortoise-tts or https://github.com/NVIDIA/tacotron2. Or here for more: https://www.reddit.com/r/MachineLearning/comments/12kjof5/d_...

Which might use https://github.com/NVIDIA/tacotron2 in their backend

One thing is uberduck.ai but I think it's freemium (it's free but some features are premium). There's also tacotron 2.0 and its pytorch page. Many other softwares on sub but tacotron gave this and this and this.

Maybe we can use AI to hear this rapped by a famous rapper?

Sadly GothicBot the TTS I knew, doesn't exist anymore, but here is an alternative. It works in polish from what I heard.

From what I’ve seen, NVIDIA’s Tacotron2 can already be used to create some pretty convincing singing.

You might want to try something like Tacotron 2 by Nvidia to experiment with your current data.

We have the TorToiSe repo, the SV2TTS repo, and from here you have the other models like Tacotron 2, FastSpeech 2, and such. A there is a lot that goes into training a baseline for these models on the LJSpeech and LibriTTS datasets. Fine tuning is left up to the user.