MultiDiffusion
auto-sd-paint-ext
MultiDiffusion | auto-sd-paint-ext | |
---|---|---|
13 | 27 | |
911 | 472 | |
- | - | |
4.8 | 3.7 | |
8 months ago | 5 months ago | |
Jupyter Notebook | Python | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
MultiDiffusion
-
Opendream: A Non-Destructive UI for Stable Diffusion
For composing this approach works pretty well
https://multidiffusion.github.io/
-
Messing with the denoising loop can allow you to reach new places in latent space. Over 8+ different research papers/Auto1111 extension ideas in a single pipe. Load once and do lots of different things (SD 2.1 or 1.5)
So I've continued to experiment with how many papers I can fit into a single pipe and have them play nicely together. The images below were created by combining the panorama code from omerbt/MultiDiffusion with the ideas from albarji/mixture-of-diffusers. Also turns out nateraw/stable-diffusion-videos can be seen as a special case of a panorama (in latent space rather than prompt space).
- MultiDiffusion Region Control, a prompt on each mask webui extension is out.
-
Hubble Diffusion with MultiDiffusion
Essentially, I fine-tuned Stable Diffusion 2.1 base (the 512x512) model on the ESA Hubble Deep Space Images & Captions dataset I collected from public Hubble images & captions. After around 33,000 training steps, I saved the model and was really impressed by the results. But I really wanted to be able to generate wallpaper-level quality space images, so I stumbled upon MultiDiffusion: a new project for generating massive panorama images using stable diffusion models. I then used hubble-diffusion-2 along with MultiDiffusion to generate each one of these amazing 2560x1536 images. Each image took a little over an hour to generate on a Google Colab T4 GPU. I used the following prompts for each of these images:
- MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation
-
What is the maximum size a 3090 24gb can produce?
If you need generated and not upscaled 4k for some reason, try something like https://github.com/omerbt/MultiDiffusion
-
[R] [N] "MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation" enables controllable image generation without any further training or finetuning of diffusion models.
Project: https://multidiffusion.github.io/ Paper: https://arxiv.org/abs/2302.08113 GitHub: https://github.com/omerbt/MultiDiffusion
-
Meet MultiDiffusion: A Unified AI Framework That Enables Versatile And Controllable Image Generation Using A Pre-Trained Text-to-Image Diffusion Model
Quick Read: https://www.marktechpost.com/2023/02/24/meet-multidiffusion-a-unified-ai-framework-that-enables-versatile-and-controllable-image-generation-using-a-pre-trained-text-to-image-diffusion-model/ Paper: https://arxiv.org/abs/2302.08113 Github: https://github.com/omerbt/MultiDiffusion Project: https://multidiffusion.github.io/
-
You to can create Panorama images 512x10240+ (not a typo) using less then 6GB VRAM (Vertorama works too). A modification of the MultiDiffusion code to pass the image through the VAE in slices then reassemble. Potato computers of the world rejoice.
So I haven't made many images with Stable Diffusion despite using it heavily. The reason is I've been messing with the internals of the diffusion pipe, to interfere with the diffusion process in different ways. Todays fun result is based on omerbt/MultiDiffusion for making panoramas.
-
First version of Stable Diffusion was released on August 22, 2022
If we combine Mixture of Diffusers + MultiDiffusion+ Composer+ cross-domain-compositing and probably some more I'm not thinking of.
auto-sd-paint-ext
-
Opendream: A Non-Destructive UI for Stable Diffusion
It makes more sense to embed stable diffusion capabilities into well-established image editors such as Gimp, Photoshop, Krita, or Figma, which come with layered, non-destructive functionalities, rather than attempting the opposite approach.
https://github.com/Interpause/auto-sd-paint-ext
-
it's so convenient
You might be interested in knowing that Krita has a SD plug in that I've found works quite well if you're willing and able to install and run SD locally. Krita is now my primary art program. https://github.com/Interpause/auto-sd-paint-ext
- Adobe just added generative AI capabilities to Photoshop 🤯
-
Why am I sharing the extensive list of alternatives to Adobe products? No reason... (I strongly personally recommend Krita.)
GitHub - Interpause/auto-sd-paint-ext: Extension for AUTOMATIC1111 to add custom backend API for Krita Plugin & more
- I've created a simple Gimp plugin that allows you to use Automatic1111's API
-
Major update: Automatic1111 Photoshop Stable Diffusion plugin V1.2.0, ControlNet, One Click Installer and More, Free and Open Source
Yep, Krita has a SD plugin too ( https://github.com/Interpause/auto-sd-paint-ext ) but probably not as good as this Photoshop one yet.
- Just discovered the auto-sd-paint-ext / Krita Extension from web-ui A111
-
If Midjourney runs Stable Diffusion, why is its output better?
Again in my humble opinion, however, it's nice to be able to choose (at least if you're willing to pay the subscription costs) among different models and algorithms. Use whatever suits best your needs or your ideas. I personally prefer SD's versatility, and the workflows enabled by "being able to get my hands on all the parameters", like sampler selection, CFG scale, inpainting and other stuff (have you tried InvokeAI's unified canvas or Automatic1111's Krita plugin?). Not to mention the ability to do progressive and potentially infinite upscaling of an image (Automatic1111's "SD upscale" script or InvokeAI's "Embiggen").
-
Trouble inpainting a subject into a scene in krita using auto-sd-paint-ext
I have a simple question that apparently i cant seem to figure out for the past like 12 hours, im using the https://github.com/Interpause/auto-sd-paint-ext auto extension for the krita plugin, and im simply just trying to inpaint creatures and just general things into a scene, but no matter what i do, the result is nothing like it, i followed the inpainting tutorial they have listed on github (New layer from visible) then make a new layer to paint the mask on, tried all fill methods with different steps, im fairly new to all of this and what not, im using a custom model checkpoint btw if that matters (protogen 5.8)
- Stable.art: open-source photoshop plugin for Automatic1111 (locally or Google Colab!) with integration of Lexica.art prompts
What are some alternatives?
stable-diffusion-webui-two-shot - Latent Couple extension (two shot diffusion port)
Auto-Photoshop-StableDiffusion-Plugin - A user-friendly plug-in that makes it easy to generate stable diffusion images inside Photoshop using either Automatic or ComfyUI as a backend.
sd-webui-controlnet - WebUI extension for ControlNet
stable-diffusion-webui - Stable Diffusion web UI
mixture-of-diffusers - Mixture of Diffusers for scene composition and high resolution image generation
IOPaint - Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
Diffusion-Models-Papers-Survey-Taxonomy - Diffusion model papers, survey, and taxonomy
InvokeAI - InvokeAI is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, supports terminal use through a CLI, and serves as the foundation for multiple commercial products.
stable-diffusion-videos - Create 🔥 videos with Stable Diffusion by exploring the latent space and morphing between text prompts
openOutpaint - local offline javascript and html canvas outpainting gizmo for stable diffusion webUI API 🐠
openpose-editor - Openpose Editor for AUTOMATIC1111's stable-diffusion-webui
krita_stable_diffusion - A Stable Diffusion plugin for Krita