speech-and-text-unity-ios-android VS TTS

PingAK9/Speech-And-Text-Unity-iOS-Android: Speed to text in Unity iOS use Native Speech Recognition (github.com)

It's really easy for a technical person to do as well.

I use Coqui TTS[0] as part of my home automation, I wrote a small python script that lets me upload a voice clip for it to clone after I got the idea from HeyWillow[1], and a small shim that lets me send the output to a Home Assistant media player instead of using their standard output device. I run the TTS container on a VM with a Tesla P4 (~£100 to buy) and get about 1x-2x (roughly the same time it'd take to say it, to process) using the large model.

Just for a giggle, I uploaded a few 3s-5s second clip of myself speaking and cloned my voice, then executed a command to our living room media player to call my wife into the room; from another room, she was 100% convinced it was myself speaking words I'd never spoken.

I tried playing with a variety of sentences for a few hours and overall, it sounded almost exactly like me, to me, with the exception of some "attitude" and "intonation" I know I wouldn't use in my speech. I didn't notice much of an improvement using much longer clips; the short ones were "good enough".

Tangentially, it really bugs me that most phone providers in the UK insist you record a "personal greeting" now before they'll let you check your voice mail box, I just record silence, because the last thing I want/need is a voicemail greeting in my voice confirming to some randomer I didn't want calling me, who I am and that my number is active, even more so knowing how I can

[0] https://github.com/coqui-ai/TTS

For anyone who is interested, CoquiTTS (formerly, MozillaTTS) was great, but the project isn't maintained anymore (athough there's been some confusion about whether or not it's active. See: https://github.com/coqui-ai/TTS/issues/4022).

Looks like there's an effort to keep an actively maintained fork here, though: https://github.com/idiap/coqui-ai-TTS

I am super impressed by the quality of voice cloning offered by Eleven Labs and Play.ai. I feel like I see impressive OSS demos on social frequently, but last weekend I took a few popular ones for a spin and quality wasn't even close to the proprietary models.

https://github.com/coqui-ai/tts

The license is the MPL, which allows commercial use?

https://github.com/coqui-ai/TTS/blob/dev/LICENSE.txt

When I did a similar thing (but with less LLM) I liked https://github.com/coqui-ai/TTS but back then I needed to cut out the conversion step from tensor to a list of numbers to make it work really nicely.

I just noticed that https://coqui.ai/ is "Shutting down".

I'm building a web app (React / Django) which takes a list of affirmations & goals (in Markdown files), puts them into a database (SQlite), and uses voice synthesis to create voice audio files of the phrases. These are combined with a relaxed backing track (ffmpeg), made into playlists of 10-20 phrases (randomly sampled, or according to a theme: "mind" "body" "soul") and then play automatically in the morning & evening (cron). This allows you to persistently hear & vocalize your own goals & good vibes over time.

I had been planning to use Coqui TTS as the local text-to-speech engine, but with this cancellation, I'd love to hear from the community what is a great open-source, local text-to-speech engine?

Generally, I learn both the highest quality commercially available technology (example: ElevenLabs), and also the best open-source equivalent. Would love to hear suggestions & perspectives on this. What voice synth tools are you investing your time into learning & building with?

lol this marketing technique is getting very old. https://github.com/coqui-ai/TTS is already amazing and open source.

Not sure how relevant this is but note that Coqui TTS (the realistic TTS) has already shut down

https://coqui.ai

I've used coqui.ai's TTS models[0] and library[1] to great success. I was able to get cloned voice to be rendered in about 80% of the audio clip length, and I believe you can also stream the response. Do note the model license for XTTS, it is one they wrote themselves that has some restrictions.

[0] https://huggingface.co/coqui/XTTS-v2

[1] https://github.com/coqui-ai/TTS