instruct-pix2pix VS stable-diffusion-webui-model-toolkit

My guess is you're thinking of InstructPix2Pix[1], with prompts like "make the sky green" or "replace the fruits with cake"

[1] https://github.com/timothybrooks/instruct-pix2pix

This comes from https://github.com/timothybrooks/instruct-pix2pix, there is also an extension to use it in Automatic1111 Stable diffusion webui.

InstructPix2Pix

Is there another place I can download the model? I tried downloading the file using the instructions on this page:

It comes from https://github.com/timothybrooks/instruct-pix2pix, you can try it out https://huggingface.co/spaces/timbrooks/instruct-pix2pix

interesting, I've been running it using this: https://github.com/timothybrooks/instruct-pix2pix/blob/main/LICENSE

This is already done and it's already been implemented in the most popular web-ui for stable diffusion too. Granted the results aren't perfect yet.

Select the script, drop in a GIF, use img2img as normal to process it. Supports quick non-ffmpeg interpolation, and works surprisingly well with InstructPix2Pix. Intended to be a fun no-nonsense GIF pipeline.

Github Issue - Closed

I think you can prune your model to under 4G using the webui-model-toolkit

I am using Model Toolkit in A1111, and most of the time it does it's job well by removing junk data and converting model to fp16. This results in 2GB file, but... There are models that cannot be converted to 2GB version. Noticeably CyberRealistic v2.0 has this "issue".

Use the model toolkit extension in A1111 -> https://github.com/arenatemp/stable-diffusion-webui-model-toolkit

I merged this on Colab, so you don't need a powerful graphics card. You'll need the block merge extension,the model toolkit extension and the supermerger extension.

If issue persists after trying the suggestions given, fix the model by using the model toolkit: https://github.com/arenatemp/stable-diffusion-webui-model-toolkit

Also, I'd highly recommend getting the Model Toolkit. It's an extension that can load up your models and prune information it doesn't need, making them drastically smaller. It can also fix broken models. You have to install it manually, but it's pretty easy (and explained on the GitHub). Well worth downloading.

Protogen Infinity has been my go to in recent weeks (I recommend getting the full unpruned version, and pruning/fixing it with arenatemp's toolkit). Very versatile, flexible, creative, diverse, high quality, etc., but I think I'll be trying Realistic Vision 1.4 (and maybe Deliberate v2) more now. I might try merging Realistic Vision and Protogen Infinity, and see if that improves on either of them (merging RV and Deliberate together didn't necessarily see any improvements).

Someone made a "model fixer" for this reason too apparently.

I agree about having to reload the UI. Since I have some pluigins installed I have to restart the UI or the engine (via CTRL-C) to maintain that 576x576 resolution as it'll run out of just enough memory to do it sometimes. I've found that safetensor format makes reloading MUCH QUICKER after a restart and I've used the Basic tab on this extension to trim the fat out of models I use, while converting them to fp16 and safetensor format.