[P] Visualizing evolution of Text-to-Image generation algorithms side by side by generating video from song-lyrics (X-LXMERT v/s AlpehImage/Dall-E)

This page summarizes the projects mentioned and recommended in the original post on /r/MachineLearning

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • pretrained-gan-minecraft

    Minecraft GAN

  • Using Minecraft GAN: https://boredhumans.com/minecraft.php (made with https://github.com/jeffheaton/pretrained-gan-minecraft) . Unlike #2 and #3 below, the images were AI-generated but not based on my lyrics.

  • Story2Hallucination

  • Using Story2Hallucination: https://boredhumans.com/music_videos/Story2Hallucination_withwords.mp4 (made with https://github.com/lots-of-things/Story2Hallucination). The problem with it was that I had no easy way to match the timing of the lyrics on the screen with the real singing. So I then made a new version at https://boredhumans.com/music_videos/Story2Hallucination_nowords.mp4 where I edited the code so it did not show the words at all. But it is somewhat boring to watch.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • deep-music-visualizer

    The Deep Visualizer uses BigGAN (Brock et al., 2018) to visualize music.

  • Using Deep Music Visualizer - https://boredhumans.com/music_videos/Big_GAN.mp4 (I made it small as a fast demo but it could easily be normal sized) . Made with https://github.com/msieg/deep-music-visualizer. Although the output is good, it uses 12 random classes (or you can pick them manually) of items from ImageNet, and the 1000 classes ImageNet uses really suck for making music videos. They don't have general categories such as beach, ocean, stars. Instead it is very specific things like moving van, drilling platform, and envelope.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts