stable-diffusion-webui-tensorrt
tomesd
| stable-diffusion-webui-tensorrt | tomesd | |
|---|---|---|
| 5 | 18 | |
| 305 | 1,220 | |
| - | - | |
| 2.9 | 5.4 | |
| 10 months ago | 6 months ago | |
| Python | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
stable-diffusion-webui-tensorrt
-
Question about automatic1111 set up after changing gpu
Also to note there is now the TensorRT to double your speed with. I have also calculated a few optimal settings for this as well. Max batch size of 7 with max height and width at 512. so 7x512x512. I have added a pull request for some gui changes and calculations for the extension which I am not sure have been added yet, but if you would like to have the gui tell you which settings for the max width, height, batch size, I can give you the code to update the extension. Also we added the ability to batch convert files from models>onnx and onnx>trt. Trts are the optimized files/models that are used to generate images 2+ times faster.
-
I converted a few popular models to TensorRT - Big Speed Improvements
Unless something has changed in the last 7 days or for some reason the official repo isn't up to date, Loras are currently not supported and would have to be baked into the model to work.
- How do you figure out the shape of a checkpoint?
-
Optimization comparison in A1111 1.3: TensorRT vs. xformers vs SDP
I just followed the instructions here: https://github.com/AUTOMATIC1111/stable-diffusion-webui-tensorrt
- A1111 just added support for TensorRT for webui as an extension!
tomesd
-
List of all the ways to improve performance for stable diffusion.
They show up to 5.4 times greater: you can see his results in the image on the github repo here: https://github.com/dbolya/tomesd
-
Question about automatic1111 set up after changing gpu
Another optimization extension you can use as well is token merging which has reported around 5.4x faster image generation.
- +39%~51% faster at the cost of some details? ToMe officially arrives to Auto1111's webui v1.3.0
-
AUTOMATIC1111 updated to 1.3.0 version
It merges redundant tokens: https://github.com/dbolya/tomesd So it can make the generation slightly faster.
-
I made some changes in AUTOMATIC1111 SD webui, faster but lower VRAM usage
Mods patched - Tomesd - Pillow-SIMD - OpenCV-CUDA (WIP) - Removed some unused imports and startup checking - Improved performance with reduced VRAM usage (tested on txt2img only) - Added a new option to use external RealESRGAN with --external-realesrgan
-
Honest question, how are people getting ~35-40 it/sec on 4090? My spits 20 at most
Were the 40 it/s perhaps achieved with ToMe?
-
Vlad diffusion keeps growing. Big thanx to all supporters :)
Done! Proposal
-
Token Merging actually works and reduces generation time as well as RAM
This feature comes from this project: https://github.com/dbolya/tomesd
-
How can I squeeze every ounce of performance from web UI?
GitHub - dbolya/tomesd: Speed up Stable Diffusion with this one simple trick!
- Token Merging for Fast Stable Diffusion
What are some alternatives?
sd-webui-deforum - Deforum extension for AUTOMATIC1111's Stable Diffusion webui
stable-diffusion-webui-ux - Stable Diffusion web UI UX
TensorRT - NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
ComfyUI - The most powerful and modular stable diffusion GUI, api and backend with a graph/nodes interface.
stable-diffusion-webui - Stable Diffusion web UI
automatic - SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
sd-extension-system-info - System and platform info and standardized benchmarking extension for SD.Next and WebUI
stable-diffusion-webui - Stable Diffusion web UI
diffusers - 🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
stable-diffusion-webui-directml - Stable Diffusion web UI
stable-diffusion-webui - Stable Diffusion web UI
AITemplate - AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.