SpecVQGAN
ru-dalle
| SpecVQGAN | ru-dalle | |
|---|---|---|
| 2 | 50 | |
| 318 | 1,639 | |
| - | -0.3% | |
| 2.2 | 0.0 | |
| 11 months ago | over 1 year ago | |
| Jupyter Notebook | Jupyter Notebook | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SpecVQGAN
-
Text-to-Audio Generation Using Instruction Tuned LLM and Latent Diffusion Model
Excellent. Some of the theory here goes back to Oct/2021 and beyond [1].
The riffusion.com [2] guys made this practical. Also, my video of high-level overview and examples [3].
1. SpecVQGAN: https://github.com/v-iashin/SpecVQGAN
2. Riffusion: ://www.riffusion.com/
3. Riffusion high-level overview: https://youtu.be/olkLVGcvib8
- "Taming Visually Guided Sound Generation". Quickly generate audio matching a given video. Code includes a Google Colab.
ru-dalle
- I trained a custom AI model for fakemon outputs. Feel free to use them for inspiration! No credit needed.
-
I trained an AI model to help me design fakebadge concepts. Full album in comments. Please feel free to take these for your own inspiration, too!
It’s a custom trained model, built in rudalle https://github.com/ai-forever/ru-dalle
-
Using AI to draft new ideas for legendaries.
It's a custom model, built from rudalle
-
SD photorealism to the extreme, is MJ really that better?
ru-dalle has had that feature for quite a while, as it was their first inpainting example notebook:
-
2 Google Colab notebooks are available for the large ruDALL-E Kandinsky model (12 billion parameters). The smaller ruDALL-E model has 1.3 billion parameters.
GitHub repo.
-
Colab notebook "pharmapsychotic modified rudalle" lets the user choose which of 4 ruDALL-E models to use
Colab notebook. There are actually 5 models, but I doubt the 12B parameter Kandinsky model is actually available per looking at this code.
-
Tree in a field.
This was made with a mini version of DALL-E: ruDALL-E
-
I trained an AI model to generate images of ancient Roman imperial denarii
Specifically, I fine-tuned ru-DALLE using a dataset consisting of ~1000 images of imperial denarii (ranging from Augustus through Maximinus Thrax) coupled with descriptions of each coin grabbed from OCRE. For example, the obverse description of this coin would be "Head of Augustus, bare, right" and the reverse description would be "Round shield, spear-head, and curved sword".
-
New ruDALL-E 1.3 billion parameter model version 3 has been released with ruDALL-E v1.0.0
One way to use the version 3 model is to use this official Colab notebook linked to in the ruDALL-E GitHub repo. I recommend making the changes mentioned in this post. If you want to use the older version 2 model with this Colab notebook, change 'Malevich' to 'Malevich_v2' in line "dalle = get_rudalle_model('Malevich', pretrained=True, fp16=True, device=device)" (relevant source code).
-
Preview of ruDALL-E v0.5.0 from the developer
# !pip install rudalle==0.0.1rc8 > /dev/null !pip3 install git+https://github.com/sberbank-ai/ru-dalle.git@feature/new_malevich
What are some alternatives?
poolformer - PoolFormer: MetaFormer Is Actually What You Need for Vision (CVPR 2022 Oral)
NeuralTextToImage - Colabs for text prompt steered image generators
vid2cleantxt - Python API & command-line tool to easily transcribe speech-based video files into clean text
pytorch-seq2seq - Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.
MoViNet-pytorch - MoViNets PyTorch implementation: Mobile Video Networks for Efficient Video Recognition;
fastai - The fastai deep learning library
awesome-python-applications - 💿 Free software that works great, and also happens to be open-source Python.
naver-webtoon-faces - Generative models on NAVER Webtoon faces
BMT - Source code for "Bi-modal Transformer for Dense Video Captioning" (BMVC 2020)
ganspace - Discovering Interpretable GAN Controls [NeurIPS 2020]
WOLOF-ASR-Wav2Vec2 - Audio Preprocessing and finetuning of wav2vec2-large-xlsr model on AI4D Baamtu Datamation - Automatic Speech Recognition in WOLOF Data.
FinRL-Meta - FinRL-Meta: Dynamic datasets and market environments for FinRL.