projected-gan VS taming-transformers

If you are looking for more traditional noise -> xxx GANs, go for https://github.com/autonomousvision/projected_gan/. Another recent work is https://github.com/nupurkmr9/vision-aided-gan.

Made with two ml algorithms, one that generates a tiny 256x256 'base' image, and another that upscales it. Both of them have been trained on a astrophotography-based dataset, but the model architecture are open sourced from Projected-Gan and Real-ESRGAN respectively.

Projected GANs are maybe the currently best model for small datasets. You could also check out StyleGAN 2 ada. https://github.com/autonomousvision/projected_gan

taming-transformers

See https://arxiv.org/abs/2012.09841 for prior work. SD authors swap out the Transformer and language modelling objective with a UNet diffusion objective. In general, the more inductive bias your model has, the more efficient it can be. ChatGPT runs purely on a Transformer architecture, which has far fewer priors than a CNN and requires far more parameters as a result. This may not be the case in the future.

I say this to everyone... Even if SD and the model is legit and legal. Do not go around commercialising it's outputs or claiming ownership over them... and if you do the properly cite the source of the model and system along with it. In https://github.com/CompVis/stable-diffusion, https://github.com/CompVis/taming-transformers and https://huggingface.co/CompVis/stable-diffusion-v1-4 there are citiations provided for you to use for a reason. I recommend you to use them.

# Copy models as described in README cp ~/Downloads/model.ckpt . cp ~/Downloads/GFPGANv1.3.pth . # Clone other repos as mentioned in README mkdir repositories git clone https://github.com/CompVis/stable-diffusion.git repositories/stable-diffusion git clone https://github.com/CompVis/taming-transformers.git repositories/taming-transformers git clone https://github.com/sczhou/CodeFormer.git repositories/CodeFormer git clone https://github.com/salesforce/BLIP.git repositories/BLIP export NIXPKGS_ALLOW_UNFREE=1 nix-shell default.nix pip install torch --extra-index-url https://download.pytorch.org/whl/cu113 # Also from linux instructions. Can probably be added to default.nix python webui.py

Pip subprocess error: ERROR: Error [WinError 2] The system cannot find the file specified while executing command git clone -q https://github.com/CompVis/taming-transformers.git 'C:\stable-diffusion\stable-diffusion-main\src\taming-transformers' ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?

bash pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip pip install -e .

You can have a look at style transfer or the recent advances in GANs (VQGAN for example). The approach you would want to try is called fine-tuning, meaning retraining an already pretraines model on your data. In terms of programming framework I guess pytorch would be the best pick. Style transfer: https://pytorch.org/tutorials/advanced/neural_style_tutorial.html VQGAN: https://github.com/CompVis/taming-transformers

MIT is a commercial license. The reference implementations of VQGAN and CLIP are both MIT licensed.