CLIP-Guided-Diffusion
feed_forward_vqgan_clip
| CLIP-Guided-Diffusion | feed_forward_vqgan_clip | |
|---|---|---|
| 4 | 4 | |
| 377 | 136 | |
| - | - | |
| 0.0 | 3.7 | |
| over 1 year ago | 4 months ago | |
| Python | Python | |
| GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
CLIP-Guided-Diffusion
-
Which is your favorite text to image model overall?
Runner-ups are Craiyon (for being more "creative" than SD), Disco Diffusion, minDALL-E, and CLIP Guided Diffusion.
-
Once have access, do you run it on your computer or over the internet on Open-AI's computers?
-clip guided diffusion https://github.com/nerdyrodent/CLIP-Guided-Diffusion
-
how would i go about running disco diffusion locally?
Nerdy Rodent has a Github repo for this; it should work fine from the Anaconda command line: https://github.com/nerdyrodent/CLIP-Guided-Diffusion
-
PLAYING AGAIN (CLIP GUIDED DIFFUSION) (VQGAN + CLIP) (Beksinski)
As far as I understand, VQGAN is not a guided diffusion model. I've been using a slightly tweaked version of https://github.com/nerdyrodent/CLIP-Guided-Diffusion for diffusion. Once you get it set up the interface is pretty much what you might expect:
feed_forward_vqgan_clip
-
[D] Hosting AI Art Generative ML Model
WOMBO I suspect uses the feed forward inferential approach to VQGAN + CLIP (instead of finetuning, predict the final z latent vector for a given text input) which is why their outputs are less sophisticated: as a result there are many deployment optimizations you can do to speed that up, which may be complicated.
-
A small experiment on how changes in a text prompt may affect output image in a CLIP-based system
The system used to produce these images is unlike most other VQGAN+CLIP systems because it uses a neural network trained by the developer(s) instead of an iterative process. This system is known to have a "formula" for image layout.
-
Get a VQGAN output image for a given text description almost instantly (not including time for one-time setup) using Colab notebook "Feed Forward VQGAN CLIP - Using a pretrained model" from mehdidc. Here are 20 non-cherry picked images from the notebook. Details in a comment.
Hello, some news. For those who are interested, I released new models (release 0.2) that you could try and you might find them better (depending on the prompt) than the current one(s), also the problem that was mentioned by /u/Wiskkey is less visible (object parts appearing systematically on top-left), but still not 100% solved, there is still a common global structure that can be identified, but it's more centered on the image. The Colab notebook was updated to use the new models.
What are some alternatives?
VQGAN-CLIP - Just playing with getting VQGAN+CLIP running locally, rather than having to use colab.
DALLE-mtf - Open-AI's DALL-E for large scale training in mesh-tensorflow.
big-sleep - A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun
disco-diffusion
deep-daze - Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun
Text-to-Image-Synthesis - Pytorch implementation of Generative Adversarial Text-to-Image Synthesis paper
vqgan-clip-app - Local image generation using VQGAN-CLIP or CLIP guided diffusion
DALLE-pytorch - Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
mindall-e - PyTorch implementation of a 1.3B text-to-image generation model trained on 14 million image-text pairs
VQGAN-CLIP-Video - Traditional deepdream with VQGAN+CLIP and optical flow. Ready to use in Google Colab.