paint-with-words-sd
Implementation of Paint-with-words with Stable Diffusion : method from eDiff-I that let you generate image from text-labeled segmentation map. (by cloneofsimo)
Rerender_A_Video
[SIGGRAPH Asia 2023] Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation (by williamyang1991)
| paint-with-words-sd | Rerender_A_Video | |
|---|---|---|
| 13 | 4 | |
| 618 | 2,895 | |
| - | - | |
| 5.2 | 8.5 | |
| about 1 year ago | 2 months ago | |
| Jupyter Notebook | Jupyter Notebook | |
| MIT License | GNU General Public License v3.0 or later |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
paint-with-words-sd
Posts with mentions or reviews of paint-with-words-sd.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-03-01.
-
paint with words with loras and multicontrolnet (will pay if needed)
I am refering to this btw: https://github.com/cloneofsimo/paint-with-words-sd
-
More control than ControlNet - code is out for MultiDiffusion Region Control, a prompt on each mask
This essentially supercharges the Nvidia eDiffi / SD paint-with-words attempts done for the same thing previously.
- "Segmentation" ControlNet preprocessor options
-
I figured out a way to apply different prompts to different sections of the image with regular Stable Diffusion models and it works pretty well.
There is stable diffusion paint with words GitHub which probably does exactly this, but no UI ever: https://github.com/cloneofsimo/paint-with-words-sd
-
What do you think will be added/created next?
personally i want to see the ediffi paint w/words stable diffusion extension https://github.com/cloneofsimo/paint-with-words-sd/commit/789419e3a34f43a1454df5a940020cfa531fbc63 that clonesofimo was working on before he stopped
- Will models have to be retrained for when this feature is eventually added into SD?
-
Paint with words (aka NVIDIA eDiff-I)
Just found there is a repo for an NVIDIA eDiff-I style img2img workflow for Stable Diffusion. For those unfamiliar, this lets you specify where parts of your text prompt should be placed in the image giving you much greater control on the composition e.g.
-
Different Models = Different prompts?
Paint-with-Words might eventually allow something along those lines, but it's a bit awkward to use now, and AFAIK you still get bleedthrough between multiple human subjects.
-
eDiff-I: A new Text-to-Image Diffusion Model with Ensemble of Expert Denoisers
someone attempted something like paint with words but I think Nvidia's version is better looking.
- Paint with words? What is next? Hope this gets to be a module in automatic 1111 soon.
Rerender_A_Video
Posts with mentions or reviews of Rerender_A_Video.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-12-10.
-
List of Stable Diffusion research softwares that I don't think gotten widespread adoption.
https://github.com/williamyang1991/Rerender_A_Video -
- FLaNK Stack for 25 September 2023
- Rerender A Video – Prisma type style transfer for Videos
-
Rerender_A_Video installer
Free one click installer for Rerender_A_Video - https://github.com/williamyang1991/Rerender_A_Video
What are some alternatives?
When comparing paint-with-words-sd and Rerender_A_Video you can also consider the following projects:
ComfyUI - The most powerful and modular stable diffusion GUI, api and backend with a graph/nodes interface.
SeaGOAT - local-first semantic code search engine
openOutpaint - local offline javascript and html canvas outpainting gizmo for stable diffusion webUI API 🐠
Kandinsky-2 - Kandinsky 2 — multilingual text2image latent diffusion model
LECO - Low-rank adaptation for Erasing COncepts from diffusion models.
FLaNK-HuggingFace-BLOOM-LLM - https://huggingface.co/bigscience/bloom into NiFi
openOutpaint-webUI-extension - direct A1111 webUI extension for openOutpaint
FLaNK-SaoPauloBrazil - FLaNK-SaoPauloBrazil
daam - Diffusion attentive attribution maps for interpreting Stable Diffusion.
GlobalMLBuildingFootprints - Worldwide building footprints derived from satellite imagery
stable-diffusion-webui-two-shot - Latent Couple extension (two shot diffusion port)
every-breath-you-take - Heart Rate Variability Training with the Polar H10 Monitor
paint-with-words-sd vs ComfyUI
Rerender_A_Video vs SeaGOAT
paint-with-words-sd vs openOutpaint
Rerender_A_Video vs Kandinsky-2
paint-with-words-sd vs LECO
Rerender_A_Video vs FLaNK-HuggingFace-BLOOM-LLM
paint-with-words-sd vs openOutpaint-webUI-extension
Rerender_A_Video vs FLaNK-SaoPauloBrazil
paint-with-words-sd vs daam
Rerender_A_Video vs GlobalMLBuildingFootprints
paint-with-words-sd vs stable-diffusion-webui-two-shot
Rerender_A_Video vs every-breath-you-take