BMT
SpecVQGAN
| BMT | SpecVQGAN | |
|---|---|---|
| 1 | 2 | |
| 220 | 318 | |
| - | - | |
| 2.9 | 2.2 | |
| about 1 year ago | 11 months ago | |
| Jupyter Notebook | Jupyter Notebook | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
BMT
-
Hey everyone! This is a project of mine that I have been working on. It is a video captioning project. This encoder decoder architecture is used to generate captions describing scene of a video at a particular event. Here is a demo of it working in real time. Check out my Github link below. Thanks!
Great project. If you want to add attention mechanism (Transformer), this paper implementation is one of the best that I have used for the same problem: https://github.com/v-iashin/BMT
SpecVQGAN
-
Text-to-Audio Generation Using Instruction Tuned LLM and Latent Diffusion Model
Excellent. Some of the theory here goes back to Oct/2021 and beyond [1].
The riffusion.com [2] guys made this practical. Also, my video of high-level overview and examples [3].
1. SpecVQGAN: https://github.com/v-iashin/SpecVQGAN
2. Riffusion: ://www.riffusion.com/
3. Riffusion high-level overview: https://youtu.be/olkLVGcvib8
- "Taming Visually Guided Sound Generation". Quickly generate audio matching a given video. Code includes a Google Colab.
What are some alternatives?
ffsubsync - Automagically synchronize subtitles with video.
poolformer - PoolFormer: MetaFormer Is Actually What You Need for Vision (CVPR 2022 Oral)
bitrate-variation-plotter - Plots a graph showing the bitrate every second or the bitrate of every GOP, for an audio or video file.
vid2cleantxt - Python API & command-line tool to easily transcribe speech-based video files into clean text
yolo_tracking - BoxMOT: pluggable SOTA tracking modules for segmentation, object detection and pose estimation models
MoViNet-pytorch - MoViNets PyTorch implementation: Mobile Video Networks for Efficient Video Recognition;
python-mpv - Python interface to the awesome mpv media player
ru-dalle - Generate images from texts. In Russian
awesome-python-applications - 💿 Free software that works great, and also happens to be open-source Python.
WOLOF-ASR-Wav2Vec2 - Audio Preprocessing and finetuning of wav2vec2-large-xlsr model on AI4D Baamtu Datamation - Automatic Speech Recognition in WOLOF Data.
nn - 🧑🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
dressing-in-order - (ICCV'21) Official code of "Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing" by Aiyu Cui, Daniel McKee and Svetlana Lazebnik