SpeechT5
PaddleSpeech
| SpeechT5 | PaddleSpeech | |
|---|---|---|
| 4 | 6 | |
| 1,044 | 10,233 | |
| 7.3% | 2.7% | |
| 7.1 | 6.8 | |
| 23 days ago | about 1 month ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SpeechT5
-
[HELP] Speech2Speech translator with speaker voice preservation
Hey! I’m doing a somewhat similar project but for TTS / voice cloning. This might not be too relevant for you but it might be one way to solve your problem. We based our project onSpeecht5 which is a multimodal setup that can take in audio or text and output audio or text. It uses speaker embeddings to handle multiple speakers, so you could use Metas S2ST to translate audio and this model to preserve the voice by doing audio to audio speech conversion. Here’s a hugging tutorial which mentions speech conversion with speecht5 https://huggingface.co/blog/speecht5
- Nvidia Text2Video
- Foundation models for speech analysis/synthesis/modification
-
[2210.03730] SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training
The idea to separate text from speech is important. Models released today: https://github.com/microsoft/SpeechT5/tree/main/SpeechUT SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training
PaddleSpeech
-
Open Source Libraries
PaddlePaddle/PaddleSpeech
- I made Lisa-nee TTS (Imai Lisa)
- project
-
is there addon that recognize speech (from video) into text?
I couldn't find any add-ons that did what you needed. I'm sorry. Maybe you could try using PaddleSpeech to see if it works for you, but it is not a Firefox add-on, it's a CLI tool.
-
Mozilla Common Voice Adds 16 New Languages and 4,600 New Hours of Speech
Ah, damn. Didn't realise.
It also looks like Baidu are now developing their Deep Speech as open source? https://github.com/PaddlePaddle/DeepSpeech
- Server-Side Audio Transcription Software
What are some alternatives?
espnet - End-to-End Speech Processing Toolkit
TTS - 🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
Awesome-Video-Diffusion - A curated list of recent diffusion models for video generation, editing, restoration, understanding, etc.
DeepSpeech - DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.
NeMo - A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
CLAP - Contrastive Language-Audio Pretraining
TensorVox - Desktop application for neural speech synthesis written in C++
AudioMAE - This repo hosts the code and models of "Masked Autoencoders that Listen".
common-voice-android - Repository of "CV Project" app. It's an unofficial app for Mozilla Common Voice, which permits you to contribute to this project via your device.
TTS - :robot: :speech_balloon: Deep learning for Text to Speech (Discussion forum: https://discourse.mozilla.org/c/tts)
kaldi-offline-transcriber - Offline transcription system for Estonian using Kaldi