Make-A-Video is a state-of-the-art AI system that generates videos from text

• NeROIC

  4 910 0.0 Python
  

I actually have been trying it out this week, and in fact it's currently trying to process the video generation, like their example shows. While I was able to follow their steps for training using their dataset, and generate the lighting/depth maps for the milkcarton example, the video generation is taking a long time (over 24hours, using a 3070Ti with 8GB VRAM).

From what I understand with NeROIC, it's not particularly meant to be able to generate an 3D model that can be imported into Blender (or other software). It requires more work to take the meshes it generates to do something with it. See https://github.com/snap-research/NeROIC/issues/10

• text2mesh

  14 906 2.1 Jupyter Notebook

  3D mesh stylization driven by a text input in PyTorch

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• Clip-Forge

  6 360 0.0 Python
  

• make-a-video-pytorch

  6 1,839 4.8 Python

  Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

Amazing. And lucidrains is on the case as well: https://github.com/lucidrains/make-a-video-pytorch

• jukebox

  129 7,571 0.0 Python

  Code for the paper "Jukebox: A Generative Model for Music"

  

> Something that can be further refined by humans is more interesting.

Exactly, that is what I was trying to say. The way I look at it is that most people who have Ableton installed cannot create an amazing song. Now let's say they are able to prompt a Stable Diffusion Audio system with a prompt like kanye type beat with flute melody in the key of E.

The system might outpu 90% hot garbage, but it's easy to skip that within seconds of hearing it. So they clip and loop the good part, add whatever personal skills they do have, and upload that.

And wow, I just found out that OpenAI's Jukebox[0] was creating this stuff two years ago. This seems like the lowest hanging fruit to me, compared to visuals. Also could be extremely lucrative. I wonder if we are already listen to ML generated music and it's just not advertised?

[0] https://openai.com/blog/jukebox/

related post: https://news.ycombinator.com/item?id=23032243

[0]

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project