deep-music-visualizer VS Story2Hallucination

Got inspired by this and started to tinker with librosa. Basically it just zooms and translates on beat.

I messed around with this "deep-music-visualizer" package a few years back. I think the concept is close to what you're talking about. It takes an MP3 as input and uses BigGAN to generate images from ImageNet classes, with the rate of change shifting with tempo and class selection with pitch.

My first attempt at using deep-music-visualizer, so what other song could I have picked ;-)

Software: https://github.com/msieg/deep-music-visualizer

Animation created using: https://github.com/HighCWu/anime_biggan_toy https://github.com/msieg/deep-music-visualizer

Hi, I'm trying to figure this out to make visuals for my music, and am totally new to Python. I've basically gotten to the point that I can at least confirm I have Python 3 on my Mac in terminal. I am trying to to run this: https://github.com/msieg/deep-music-visualizer

I'm a film major and I want to learn how to use Deep Music Visualizer (https://github.com/msieg/deep-music-visualizer) I installed Python 3.9.5 but I dont understand how to even run the visualizer. If anyone has used this program before, please let me know if this is familiar

I am wondering how is this different from Deep Music Visualizer?

Using Deep Music Visualizer - https://boredhumans.com/music_videos/Big_GAN.mp4 (I made it small as a fast demo but it could easily be normal sized) . Made with https://github.com/msieg/deep-music-visualizer. Although the output is good, it uses 12 random classes (or you can pick them manually) of items from ImageNet, and the 1000 classes ImageNet uses really suck for making music videos. They don't have general categories such as beach, ocean, stars. Instead it is very specific things like moving van, drilling platform, and envelope.

(Added Feb. 5, 2021) Story2Hallucination.ipynb - Colaboratory by bonkerfield. Uses BigGAN to generate images/videos. GitHub.

Third which is pretty new to me is the Story2Hallucination. Which takes text, (say your story) and uses Google Deep Sleep to create a visual using AI to generate images for what the story is describing. And example can be found here.

I ran some Lovecraft through Story2Hallucination[1] which uses Big Sleep to make videos from text.

The results were quite something - https://m.imgur.com/tfWLsSR

[1] https://github.com/lots-of-things/Story2Hallucination

Using Story2Hallucination: https://boredhumans.com/music_videos/Story2Hallucination_withwords.mp4 (made with https://github.com/lots-of-things/Story2Hallucination). The problem with it was that I had no easy way to match the timing of the lyrics on the screen with the real singing. So I then made a new version at https://boredhumans.com/music_videos/Story2Hallucination_nowords.mp4 where I edited the code so it did not show the words at all. But it is somewhat boring to watch.

Here you go: https://github.com/lots-of-things/Story2Hallucination/

This is basic attempt at that: https://github.com/lots-of-things/Story2Hallucination . It turns the text you write into a dream-like series of images converted into an animated GIF. It does not look real, but it is a start.

Story2Hallucination github by u/bonkerfield : GITHUB LINK

I've added a slightly simpler notebook to Story2Hallucination that can render GIFs on Google Colab. Note that the story text has to be fairly short or it will make a gigantic unrenderable GIF.

Credit to https://github.com/lots-of-things/Story2Hallucination/ and onwards - I converted the script to run in regular python, and on windows through WSL+CUDA (Though, the windows tweaks seem to have caused other issues, Will probably have to roll back and dualboot on this device to do more)