InvokeAI VS ControlNet

Probably not, since I have no idea what you're talking about. I've just been using the models that InvokeAI (2.3, I only just now saw there's a 3.0) downloads for me [0]. The SD1.5 one is as good as ever, but the SD2 model introduces artifacts on (many, but not all) faces and copyrighted characters.

[0] https://github.com/invoke-ai/InvokeAI

I actually used the rocm/pytorch image you also linked.

I'm not sure what you're pointing to with your reference to the Fedora-based images. I'm quite happy with my NixOS install and really don't want to switch to anything else. And as long as I have the correct kernel module, my host OS really shouldn't matter to run any of the images.

And I'm sure it can be made to work with many base images, my point was just that the dependency management around pytorch was in a bad state, where it is extremely easy to break.

> Anyways, hopefully this PR fixes the immediate issue: https://github.com/invoke-ai/InvokeAI/pull/5714/files

It does! At least for me. It is my PR after all ;)

The data being flagged is not in that github repo, it's fetched from elsewhere and I don't fancy spending time looking for it. The alert is for 'Sirefef!cfg' which has been reported as a false positive with a bunch of other stable diffusion projects (https://www.reddit.com/r/StableDiffusion/comments/101zjec/trojanwin32sirefefcfg_an_apparently_common_false/, https://www.reddit.com/r/StableDiffusion/comments/xmhukb/trojan_in_waifudiffusion_model_file/, https://github.com/invoke-ai/InvokeAI/issues/2773 )

I haven’t tried it recently, but InvokeAI runs on Mac. Invoke. I used to run on my MacBook, but have since gotten a Win laptop.

There are already a number of local, inference options that are (crucially) open-source, with more robust feature sets.

And if the defense here is "but Auto1111 and Comfy don't have as user-friendly a UI", that's also already covered. https://github.com/invoke-ai/InvokeAI

https://github.com/invoke-ai/InvokeAI should work on your machine. For LLM models, the smaller ones should run using llama.cpp, but I don't think you'll be happy comparing them to ChatGPT.

Stable Diffusion. /r/stablediffusion There are many tutorials on how to set it up locally and use it. InvokeAI is the easiest way to set it up. https://github.com/invoke-ai/InvokeAI

InvokeAI

It`s all possible. Have a look into Automatic1111`s Web UI, ControlNet, OpenPose and, if you don`t have a dedicated GPU with at least 8GB of VRAM, or at least 16GB of RAM to use the CPU, you can also use Stable Horde to use the webUI with a peer-to-peer connection, where you`ll only use a fraction of your resources, but you`ll be able to use local AI models with all the bells and whistles that you won`t get from "state-of-the-art" paid services.

Next up is ControlNet. Controlnet, as Illyasviel--creator of controlnet--describes it, "let's us control diffusion models!." ControlNet is a neural network structure to control diffusion models by adding extra connections. [8]. There is more to that than what I described, but the big take-away is that ControlNet takes a preprocessed image that you provide (or is generated) and uses that as a way of constraining the output the sampler's noise generates, allowing you to have a bit more control of the output. ControlNet is typically used for character or scene "artwork", which previously would have been a challenge with just prompting alone (at least with this current architecture).

For your first part of the comment, I can simply refer you to technologies like ControlNet, LoRA and prompt embedding: https://github.com/lllyasviel/ControlNet https://github.com/microsoft/LoRA

Here you can see what every controlnet preprocessor and model do, to give you an idea of ​​how to use

Ya, it takes awhile to really start getting comfortable with the wonkiness. If you are trying to do something specific, look for a LoRA, but in general I'd recommend you get controlnet so you can feed it a reference image. Another simple trick is to edit the image a bit in GIMP or a photo editor to get the color scheme you like and then feed it back to img2img at low denoising (0.1-0.2) to refine it. You can also add just garishly bad cartoon drawing or photoshop in assets and img2img will usually make something of them and blend them into your image, I find this easier than using img2img scribble.

If there are no errors, go to the "Extensions" tab, then "Install from URL". There, enter "https://github.com/lllyasviel/ControlNet" then click "Install".

This is a good place to start reading. Given the open-source nature of SD, there are setups of various difficulty available. A1111 is the "standard" people enjoy because it's easy to plug in new stuff (ControlNet, new models, etc.), but it's not inherently easy to set up and get going. There is an installer for it, but I haven't tried it.