Like Diffusion but Faster: The Paella Model for Fast Image Generation

1. Paella

   1 6 746 4.7 Jupyter Notebook

   Official Implementation of Paella https://arxiv.org/abs/2211.07292v2

   

   Github for those looking for the code https://github.com/dome272/Paella

2. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. Wuerstchen

   2 1 555 5.9 Jupyter Notebook

   Official implementation of Würstchen: Efficient Pretraining of Text-to-Image Models

   

   Fully correct, also the v2 of the paper introduced a model that is bigger and slower, however generates better images. So the 500ms was only for the first model we introduced in v1. I also want to mention our new work as it is very much related to this whole topic of "speeding up models" -> either training or sampling: Würstchen: https://github.com/dome272/wuerstchen/

4. Pytorch

   3 423 102,257 10.0 Python

   Tensors and Dynamic neural networks in Python with strong GPU acceleration

   

   - The gain in stable diffusion is modest (15%-25% last I checked?)

   - Torch 2.0 only supports static inputs. In actual usage scenarios, this means frequent lengthy recompiles. Eventually, these recompiles will overload the compilation cache and torch.compile will stop functioning.

   - Some common augmentations (like TomeSD) break compilation, make it take forever, or kill the performance gains.

   - Other miscellaneous bugs (like freezing the Python thread and causing timeouts in web UIs, or errors with embeddings)

   - Dynamic input in Torch 2.1 nightly fixes a lot of these issues, but was only maybe working a week ago? See https://github.com/pytorch/pytorch/issues/101228#issuecommen...

   - TVM and AITemplate have massive performance gains. ~2x or more for AIT, not sure about an exact number for TVM.

   - AIT supported dynamic input before torch.compile did, and requires no recompilation after the initial compile. Also, weights (models and LORAs) can be swapped out without a recompile.

   - TVM supports very performant Vulkan inference, which would massively expand hardware compatibility.

   Note that the popular SD Web UIs don't support any of this, with two exceptions: VoltaML (with WIP AIT support) and a the Windows DirectML fork of A1111 (which uses optimized ONNX models, I think).

Suggest a related project