AITemplate VS stable-diffusion-tensorflow

VoltaML is a relatively vanilla diffusers-based backend, so its not a hairy monster to hack like you may have seen with SAI-based UIs.

The AITTemplate code is a lightly modified version of Facebook's example, code, to get rid of small issues like VRAM spikes: https://github.com/facebookincubator/AITemplate/tree/main/ex...

InvokeAI is also diffusers based, but they seem to mess with the pipeline a bit more.

And anyway, all that may be a better reference for interesting features rather than a backend to try and adopt.

let me know if you discover any more ways to improve SD. I am currently looking into facebooks AITemplate : https://github.com/facebookincubator/AITemplate

Link to AITemplate itself: https://github.com/facebookincubator/AITemplate

was curious if someone attempted the conversion? I tried here https://github.com/divamgupta/stable-diffusion-tensorflow/issues/58 but having some input shapes error. First time trying the conversion here, would love to run it on a edge tpu.

Checking here is someone tried to convert the tensorflow diffusion model into a tf lite?https://github.com/divamgupta/stable-diffusion-tensorflow/issues/58

Actually I was just on GitHub trying to submit issues related to me testing Intel's PyTorch and Tensorflow extensions when I saw this; it seems that someone has already ported SD over to the tensorflow framework and so you can probably start using intel's extension for tensorflow with it immediately; and according to this article you can use Intel's extension within WSL under windows as well. But unfortunately given how the guy whose issue I linked to has been facing pretty serious performance issues of inferencing taking many minutes longer than it should when using an A770 to do SD-related inferencing, you might be better off waiting for intel's extension for tensorflow versions 1.2 and greater or something like that, so that when it's your turn to use it, Intel has already ironed out most of the major bugs within the software :)

tensorflow-stable-diffusion

# clone git clone https://github.com/divamgupta/stable-diffusion-tensorflow.git cd stable-diffusion-tensorflow # create venv python -m venv --prompt sdtf-windows-directml venv venv\Scripts\activate # verify venv is installed and activated pip --version # install deps pip install -r requirements.txt pip install tensorflow-directml-plugin # you should see DML debug output and at least one GPU python -c 'import tensorflow as tf; print(tf.config.list_physical_devices())' # run (show help) python text2image.py --help python text2image.py --prompt "a fluffy kitten"

(Answers based on stable-diffusion open model) If you have a M1 processor: https://github.com/divamgupta/diffusionbee-stable-diffusion-ui (I've tested it) Or this claimed faster with TensorFlow: https://github.com/divamgupta/stable-diffusion-tensorflow

Added inpainting support to the original keras implementation: https://github.com/divamgupta/stable-diffusion-tensorflow Colab: https://colab.research.google.com/drive/1Bf-bNmAdtQhPcYNyC-guu0uTu9MYYfLu Github page: https://github.com/ShaunXZ/stable-diffusion-tensorflow

I wonder if you mean 13 seconds per image because this implementation reports ~10s per image with mixed precision.

On intel MacBookPro, CPU-only, the original one[1] using pytorch only utilized one core. A tensorflow implementation[2] with oneDNN support which utilized most of the cores ran at ~11sec/iteration. Another OpenVINO based implementation[3] ran at ~6.0sec/iteration.

[1] https://github.com/CompVis/stable-diffusion/

[2] https://github.com/divamgupta/stable-diffusion-tensorflow/

[3] https://github.com/bes-dev/stable_diffusion.openvino/