WaveRNN VS TTS

I've posted before, but its this open source https://github.com/fatchord/WaveRNN with a very basic gui and a connection to the streamlabs socket api which sends out json of the dono. The actual app forsen has is probably a day of work.

Its not enough. You need at least 8GB VRAM for GPU training. My specs are 3700x + 2080s, CPU training is <1step/sec and GPU is up to 5steps/sec. Even with this GPU I have "out of memory" erros sometimes (hint: if someone else wants to try wavernn check my pull request to reduce gpu memory usage). You can use online services to train your data like Google Colab but it might be too expensive for you.

I may have found an a program that works. Its the same algorithm that the streamer forsen uses for his TTS, and it can emulate his own voice(which has a swedish accent) or voices of other notable people with accents. It appears like it has the ability to give it your own data set so I'll see if I can get it to work. For anyone finding the post this is the link: https://github.com/fatchord/WaveRNN

I just noticed that https://coqui.ai/ is "Shutting down".

I'm building a web app (React / Django) which takes a list of affirmations & goals (in Markdown files), puts them into a database (SQlite), and uses voice synthesis to create voice audio files of the phrases. These are combined with a relaxed backing track (ffmpeg), made into playlists of 10-20 phrases (randomly sampled, or according to a theme: "mind" "body" "soul") and then play automatically in the morning & evening (cron). This allows you to persistently hear & vocalize your own goals & good vibes over time.

I had been planning to use Coqui TTS as the local text-to-speech engine, but with this cancellation, I'd love to hear from the community what is a great open-source, local text-to-speech engine?

Generally, I learn both the highest quality commercially available technology (example: ElevenLabs), and also the best open-source equivalent. Would love to hear suggestions & perspectives on this. What voice synth tools are you investing your time into learning & building with?

lol this marketing technique is getting very old. https://github.com/coqui-ai/TTS is already amazing and open source.

Not sure how relevant this is but note that Coqui TTS (the realistic TTS) has already shut down

https://coqui.ai

I've used coqui.ai's TTS models[0] and library[1] to great success. I was able to get cloned voice to be rendered in about 80% of the audio clip length, and I believe you can also stream the response. Do note the model license for XTTS, it is one they wrote themselves that has some restrictions.

[0] https://huggingface.co/coqui/XTTS-v2

[1] https://github.com/coqui-ai/TTS

My only exposure to Coqui was their text to speech software. If I remember correctly the website was a commercialized service with TTS and probably some other related things. I hope the software work continues in the open.

https://github.com/coqui-ai/TTS

Coqui.ai TTS (https://github.com/coqui-ai/TTS)

"tts_models/multilingual/multi-dataset/xtts_v2" model from https://github.com/coqui-ai/TTS. It gives pretty good results and works with references, so it's pretty easy to change the voice. By the way the source code of the project is open: https://github.com/volotat/Anagnorisis but be ready, the code is pretty raw for now.

A recent update to their GitHub also has a no-code gradio ui to facilitate fine-tuning and inferencing locally. https://github.com/coqui-ai/TTS/releases/tag/v0.21.3