Next frame prediction with ControlNet

• ControlNet

  127 27,882 4.1 Python

  Let us control diffusion models!

It seems like a reasonable step forward to train control net to predict next frame from previous one. That should eliminate all major issues with video stabilisation and allow at least some way to do text2video generation. The training procedure is also well described in the ControlNet repository: https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md . But the fact that it wasn't done yet buggles me a lot. There must be a reason nobody done it yet. Has anybody tried to train ControlNet? Is there any merit to this approach?

• RVD

  1 54 10.0 Python

  Diffusion Probabilistic Modeling for Video Generation

Some resources I found: rvd, Next_Frame_Prediction, Next-Frame-Prediction

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Next_Frame_Prediction

  1 3 10.0 Jupyter Notebook

  Predicts the next frame of a series of gifs.

Some resources I found: rvd, Next_Frame_Prediction, Next-Frame-Prediction

• Next-Frame-Prediction

  2 3 10.0 HTML

Some resources I found: rvd, Next_Frame_Prediction, Next-Frame-Prediction

• latentblending

  17 315 8.7 Python

  Create butter-smooth transitions between prompts, powered by stable diffusion

  

latent blending: https://github.com/lunarring/latentblending This one has a lot of potential, since it can be used to transition from one prompt to the next. It interpolates between the prompst and also the image latents themselves.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project