Make-A-Video is a state-of-the-art AI system that generates videos from text

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Collect and Analyze Billions of Data Points in Real Time
  • Mergify - Updating dependencies is time-consuming.
  • Sonar - Write Clean Python Code. Always.
  • NeROIC

    I actually have been trying it out this week, and in fact it's currently trying to process the video generation, like their example shows. While I was able to follow their steps for training using their dataset, and generate the lighting/depth maps for the milkcarton example, the video generation is taking a long time (over 24hours, using a 3070Ti with 8GB VRAM).

    From what I understand with NeROIC, it's not particularly meant to be able to generate an 3D model that can be imported into Blender (or other software). It requires more work to take the meshes it generates to do something with it. See https://github.com/snap-research/NeROIC/issues/10

  • text2mesh

    3D mesh stylization driven by a text input in PyTorch

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  • Clip-Forge

  • make-a-video-pytorch

    Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

    Amazing. And lucidrains is on the case as well: https://github.com/lucidrains/make-a-video-pytorch

  • jukebox

    Code for the paper "Jukebox: A Generative Model for Music"

    > Something that can be further refined by humans is more interesting.

    Exactly, that is what I was trying to say. The way I look at it is that most people who have Ableton installed cannot create an amazing song. Now let's say they are able to prompt a Stable Diffusion Audio system with a prompt like kanye type beat with flute melody in the key of E.

    The system might outpu 90% hot garbage, but it's easy to skip that within seconds of hearing it. So they clip and loop the good part, add whatever personal skills they do have, and upload that.

    And wow, I just found out that OpenAI's Jukebox[0] was creating this stuff two years ago. This seems like the lowest hanging fruit to me, compared to visuals. Also could be extremely lucrative. I wonder if we are already listen to ML generated music and it's just not advertised?

    [0] https://openai.com/blog/jukebox/

    related post: https://news.ycombinator.com/item?id=23032243

    [0]

  • Mergify

    Updating dependencies is time-consuming.. Solutions like Dependabot or Renovate update but don't merge dependencies. You need to do it manually while it could be fully automated! Add a Merge Queue to your workflow and stop caring about PR management & merging. Try Mergify for free.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts