espnet
DeepSpeech
Our great sponsors
espnet | DeepSpeech | |
---|---|---|
15 | 67 | |
7,769 | 24,086 | |
2.4% | 1.3% | |
10.0 | 0.0 | |
about 20 hours ago | about 1 month ago | |
Python | C++ | |
Apache License 2.0 | Mozilla Public License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
espnet
-
WhisperSpeech – An Open Source text-to-speech system built by inverting Whisper
You might check out this list from espnet. They list the different corpuses they use to train their models sorted by language and task (ASR, TTS etc):
-
[D] What's stopping you from working on speech and voice?
- https://github.com/espnet/espnet
-
Text to speech generation
This work is made possible by the excellent advancements in text to speech modeling. ESPnet is a great project and should be checked out for more advanced and a wider range of use cases. This pipeline was also made possible by the great work from espnet_onnx in building a framework to export models to ONNX.
-
[P] TorToiSe - a true zero-shot multi-voice TTS engine
CMU WavLab has ESPNet https://espnet.github.io/espnet/ which includes a number of high quality TTS models including VITS (which in my subjective experience is just as good as what is demonstrated here). Also the inference on various ESPNet pretrained TTS models is reasonable and sentences take on average 5 seconds per word to generate the waveform on my totally mid PC setup.
-
Help picking a good speech recognition library
https://github.com/espnet/espnet (kind of like a newer Kaldi, but also not beginner friendly)
-
speechbrain VS espnet - a user suggested alternative
2 projects | 13 Oct 2021
both provide e2e ASR support but espnet does have more utilities where as speechbarain is clean
-
Need help with training ASR model from scratch.
This is relatively small amount of speech to train the model from scratch, but you can train using another pre-trained model for initialization. There are numbers of end-to-end ASR toolkits which can be used for this: https://github.com/NVIDIA/NeMo and https://github.com/espnet/espnet
You actually dont need to have phone level alignment for your data. Both hybrid and end-2-end approaches can work with utterance level alignment. For the hybrid approach, you would need a lexicon which maps each unique word in your training transcription to its phone sequence. You can obtain this with CMU's tool. For end-2-end approach you will need a byte pair encoder to tokenize the words in the transcriptions to its sub-words.
-
Is there a python based speaker diarization system you would recommend?
Have a look at this PR at ESPnet. It might be useful.
-
What are some good speech recognition papers I can implement?
espnet
DeepSpeech
- Common Voice
- Ask HN: Speech to text models, are they usable yet?
-
Looking to recreate a cool AI assistant project with free tools
- [DeepSpeech](https://github.com/mozilla/DeepSpeech) rather than Whisper for offline speech-to-text
I came across a very interesting [project]( (4) Mckay Wrigley on Twitter: "My goal is to (hopefully!) add my house to the dataset over time so that I have an indoor assistant with knowledge of my surroundings. It’s basically just a slow process of building a good enough dataset. I hacked this together for 2 reasons: 1) It was fun, and I wanted to…" / X ) made by Mckay Wrigley and I was wondering what's the easiest way to implement it using free, open-source software. Here's what he used originally, followed by some open source candidates I'm considering but would love feedback and advice before starting: Original Tools: - YoloV8 does the heavy lifting with the object detection - OpenAI Whisper handles voice - GPT-4 handles the “AI” - Google Custom Search Engine handles web browsing - MacOS/iOS handles streaming the video from my iPhone to my Mac - Python for the rest Open Source Alternatives: - [ OpenCV](https://opencv.org/) instead of YoloV8 for computer vision and object detection - Replacing GPT-4 is still a challenge as I know there are some good open-source LLms like Llama 2, but I don't know how to apply this in the code perhaps in the form of api - [DeepSpeech](https://github.com/mozilla/DeepSpeech) rather than Whisper for offline speech-to-text - [Coqui TTS](https://github.com/coqui-ai/TTS) instead of Whisper for text-to-speech - Browser automation with [Selenium](https://www.selenium.dev/) instead of Google Custom Search - Stream video from phone via RTSP instead of iOS integration - Python for rest of code I'm new to working with tools like OpenCV, DeepSpeech, etc so would love any advice on the best way to replicate the original project in an open source way before I dive in. Are there any good guides or better resources out there? What are some pitfalls to avoid? Any help is much appreciated!
-
Are there any secure and free auto transcription software ?
If you're not afraid to get a little technical, you could take a look at mozilla/DeepSpeech (installation & usage docs here).
-
Web Speech API is (still) broken on Linux circa 2023
There is a lot of TTS and SST development going on (https://github.com/mozilla/TTS; https://github.com/mozilla/DeepSpeech; https://github.com/common-voice/common-voice). That is the only way they work: Contributions from the wild.
-
Mozilla Launches Responsible AI Challenge
Mozilla did release DeepSpeech[0] and Firefox Translation[1] (the latter of which they included in Firefox, to offer client-side webpage translations.)
They definitely have fewer resources than OpenAI, and they do not produce SOTA research (their publications have plummeted to 1/year anyway[2]). So the only way for them to make progress is to seek government grants or make challenges like these.
This challenge is unlikely to be profitable for the winning team: the expected value of winnings are likely around $1K when taking into account the probability that another team gets a better rank, but ML research projects are often more expensive (recently, Alpaca spent upwards of $600 on computation alone; and of course pretraining large models is much more expensive). So the main gain will be publicity.
[0]: https://github.com/mozilla/deepspeech
-
Browserule
Unfortunately, only Chrome supports the technology required to provide this feature (for now). Firefox is working to include it in the browser, but it is a complex feature that requires a lot of development. Mozilla (the company who developed Firefox) actually have a tool called DeepSpeech to use speech-to-text dictation without using the Internet. I don't know if it will help you, but I've done what I could :'(
- speech-to-text on Linux?
-
Show HN: State-of-the-Art German Speech Recognition in 284 lines of C++
I wrote "284 lines of C++" to indicate that this is compact enough for people to actually read and understand the source code. Also, compiling my implementation is super easy and straightforward ... something which can't be said for Kaldi, Vosk, or DeepSpeech.
If you try to read the CTC beam search decoder from Mozilla's DeepSpeech [1], that alone is about 2000 LOC in multiple files.
If you try to read the pyctcdecode source that is used by HuggingFace [2], that's 1000+ LOC of Python.
But this implementation is all the client-side, i.e. the entire "native_client" folder hierarchy in DeepSpeech [3], narrowed down to a mere 284 lines.
[1] https://github.com/mozilla/DeepSpeech/tree/master/native_cli...
[2] https://github.com/kensho-technologies/pyctcdecode
[3] https://github.com/mozilla/DeepSpeech/tree/master/native_cli...
What are some alternatives?
Kaldi Speech Recognition Toolkit - kaldi-asr/kaldi is the official location of the Kaldi project.
speechbrain - A PyTorch-based Speech Toolkit
NeMo - NeMo: a framework for generative AI
picovoice - On-device voice assistant platform powered by deep learning
STT - 🐸STT - The deep learning toolkit for Speech-to-Text. Training and deploying STT models has never been so easy.
TTS - 🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
k2 - FSA/FST algorithms, differentiable, with PyTorch compatibility.
PaddleSpeech - Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text frontend, Speaker Verification System, End-to-End Speech Translation and Keyword Spotting. Won NAACL2022 Best Demo Award.
fairseq - Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
kaldi-gstreamer-server - Real-time full-duplex speech recognition server, based on the Kaldi toolkit and the GStreamer framwork.
dicio-android - Dicio assistant app for Android