clipseg VS imaginAIry

Following the comments to this old post, I tried to use in-painting with manual mask selection. I didn't get beautiful results but I'm sure with some tweaking, I could make it better. The main problem I had was having to manually select the area where I wanted to place the logo and trying to resize my logo mask to the fit the segment. I tried some automatic segmentation tools (Clipseg and Segment Anything). I couldn't tell the segmentation models to find a good area to for logo placement (i.e. some small flat surface). Given the complexity of what I was dealing with, I think there could be a better way (XY problem).

The addon utilizes clipseg for region masking, which was trained on "an extended version of the PhraseCut dataset" (many thousands of images.)

You'll also need to make sure to update your clipseg repo. The script won't do this for you. Namely you just need to update this models/clipseg.py file to ensure your clipseg has support for the new model.

For localisation at search time I ended up using OWL-ViT. This worked really well. I did not try Detic or CLIPseg but would be interested to hear if anyone else has tried these?

clipseg is an image segmentation method used to find a mask for an image from a prompt. I implemented it as an executor for dalle-flow and added it to my bot yasd-discord-bot.

Automated Replacement (txt2mask) by clipseg

Stable Diffusion works fine on a CPU - on an AMD Ryzen 5700, approx 90s per image (and I believe comparable or faster on my old i7-6700). If you want to kick off a batch in the background while you work on something else, that's plenty fast. (I use: https://github.com/brycedrennan/imaginAIry).

Yeah, also, rather than messing around with the AUTO1111 UI, maybe learn some Python and how to use this library: https://github.com/brycedrennan/imaginAIry

this is a real thing https://github.com/brycedrennan/imaginAIry