Use deep fake tech to say stuff with your favorite characters

• StarGANv2-VC

  3 454 1.3 Python

  StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion

This looks like it was previously known as Vocodes, made by echelon who is here on HN:

https://news.ycombinator.com/item?id=23965787

The code repos used are listed in their credits section, and it looks like a mixture of (customised?) Tacotron2, Glow-TTS, HifGan, and others. Videos are generated using Wav2Lip.

Text-To-Speech (TTS) has improved greatly over the past several years, but there's still a lot of metallic sounds in "pure" TTS implementations. I've started exploring voice style conversion, otherwise known as "voice cloning", and there are some interesting repos out there with decent results. These work differently from TTS, in that you don't type out the text to be spoken, but rather pass in an audio file of what you want the cloned speaker to say, and the system outputs an audio file with the same sounds (words, intonation) but with a different speaker identity.

This may be easier to get the right cadence and emotion in the generated audio, as text doesn't capture proper emotion and intonation. I suspect game character audio will use more of voice-style conversion instead of pure TTS simply to get the right emotional cadence of the lines being delivered.

Some interesting voice style conversion repos (in no order, just a random selection if anyone is interested in exploring):

https://github.com/yl4579/StarGANv2-VC

• voice_conversion

  1 123 3.7 Python

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• tt-vae-gan

  4 64 1.8 Python

  Timbre transfer with variational autoencoding and cycle-consistent adversarial networks. Able to transfer the timbre of an audio source to that of another.

• autovc

  1 961 2.7 Python

  AutoVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss

• YourTTS

  3 824 1.6 Jupyter Notebook

  YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone

• jukebox

  129 7,563 0.0 Python

  Code for the paper "Jukebox: A Generative Model for Music"

  

JukeBox did that for music so it's not impossible ... https://openai.com/blog/jukebox/

Suggest a related project