WhisperSpeech
EmotiVoice
| WhisperSpeech | EmotiVoice | |
|---|---|---|
| 5 | 5 | |
| 3,417 | 6,369 | |
| 4.7% | - | |
| 9.2 | 8.9 | |
| 6 days ago | 3 months ago | |
| Jupyter Notebook | Python | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
WhisperSpeech
-
OpenVoice: Versatile Instant Voice Cloning
I haven't tried openvoice, but I did try whisperspeech and it will do the same thing. You can optionally pass in a file with a reference voice, and the tts uses it.
https://github.com/collabora/whisperspeech
I found it to be kind of creepy hearing it in my own voice. I also tried a friend of mine who had a french canadian accent and strangely the output didn't have his accent.
-
Show HN: WhisperFusion – Ultra-low latency conversations with an AI chatbot
- WhisperSpeech for the text-to-speech - https://github.com/collabora/WhisperSpeech
and an LLM (phi-2, Mistral, etc.) in between
-
WhisperFusion: Ultra-low latency conversations with an AI chatbot
Hi, I used the [WhisperSpeech](https://github.com/collabora/WhisperSpeech) model for the TTS part after I did some serious torch.compile optimizations to bring the latency down. The Whisper speech recognition and the LLM were optimized through TensorRT-LLM by Marcus and Vineet.
It's not perfect but I am still extremely proud of how it came out. :)
- WhisperSpeech – An Open Source text-to-speech system built by inverting Whisper
-
StyleTTS2 – open-source Eleven Labs quality Text To Speech
I think you’re talking about just using Whisper to annotate audio for a TTS pipeline but someone from Collabora actually created a TTS model directly from Whisper embeddings https://github.com/collabora/WhisperSpeech
EmotiVoice
- FLaNK Stack Weekly 12 February 2024
-
WhisperSpeech – An Open Source text-to-speech system built by inverting Whisper
Interested to see how it performs for Mandarin Chinese speech synthesis, especially with prosody and emotion. The highest quality open source model I've seen so far is EmotiVoice[0], which I've made a CLI wrapper around to generate audio for flashcards.[1] For EmotiVoice, you can apparently also clone your own voice with a GPU, but I have not tested this.[2]
[0] https://github.com/netease-youdao/EmotiVoice
[1] https://github.com/siraben/emotivoice-cli
[2] https://github.com/netease-youdao/EmotiVoice/wiki/Voice-Clon...
-
Microsoft releases Windows AI studio to run and fine tune models locally
Interesting. I'll have to check to be sure, but I think maybe something is happening automagically if you have reasonably up to date nvidia drivers on the host OS, because I was able to run the EmotiVoice TTS docker (which requires nvidia gpu) from WSL2.
https://github.com/netease-youdao/EmotiVoice
- FLaNK Stack Weekly for 13 November 2023
- EmotiVoice: A Multi-Voice and Prompt-Controlled TTS Engine
What are some alternatives?
piper - A fast, local neural text to speech system
Cgml - GPU-targeted vendor-agnostic AI library for Windows, and Mistral model implementation.
WhisperFusion - WhisperFusion builds upon the capabilities of WhisperLive and WhisperSpeech to provide a seamless conversations with an AI.
TTS - 🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
whisper-ctranslate2 - Whisper command line client compatible with original OpenAI client based on CTranslate2.
draw-a-ui - Draw a mockup and generate html for it
monotonic_align - Monotonic Alignment Search
MockingBird - 🚀AI拟声: 5秒内克隆您的声音并生成任意语音内容 Clone a voice in 5 seconds to generate arbitrary speech in real-time
VoiceCraft - Zero-Shot Speech Editing and Text-to-Speech in the Wild
lhotse - Tools for handling speech data in machine learning projects.
whisper - Robust Speech Recognition via Large-Scale Weak Supervision
voice100 - Voice100 includes neural TTS/ASR models. Inference of Voice100 is low cost as its models are tiny and only depend on CNN without autoregression.