StyleCLIP
clip-glass
StyleCLIP | clip-glass | |
---|---|---|
23 | 13 | |
3,899 | 177 | |
- | - | |
0.0 | 0.0 | |
11 months ago | over 2 years ago | |
HTML | Python | |
MIT License | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
StyleCLIP
-
A History of CLIP Model Training Data Advances
While CLIP on its own is useful for applications such as zero-shot classification, semantic searches, and unsupervised data exploration, CLIP is also used as a building block in a vast array of multimodal applications, from Stable Diffusion and DALL-E to StyleCLIP and OWL-ViT. For most of these downstream applications, the initial CLIP model is regarded as a “pre-trained” starting point, and the entire model is fine-tuned for its new use case.
-
[D] What is the largest / most diverse GAN model currently out there?
I'm currently building a fork for StyleCLIP global directions which allows you to control multiple semantic parameters simoultaneously to generate and edit an image with StyleGAN and CLIP in realtime. I want to showcase its potential as a design tool. Unfortunately, GAN weights are trained on very domain-specific (faces, cars, churches) data. This makes them inferior to modern diffusion models which I can use to generate whatever comes to mind. Although I know we won't have a GAN-based DALL-E counterpart anytime soon, I still would love to use my system with weights that can output a wide variety of things.
-
test
(Added Feb. 15, 2021) StyleCLIP - Colaboratory by orpatashnik. Uses StyleGAN to generate images. GitHub. Twitter reference. Reddit post.
- I am David Bau, and I study the structure of the complex computations learned within deep neural networks.
-
Dragon Age Origins Companions as Photorealistic People.
I used StyleCLIP. I purchased some Google Colab time to use their GPUs. I'll probably do some more later this week.
- Turning BDO characters into blursed people with AI
-
I used AI to generate real life for honor character faces
Link for Styleclip
-
AI-generated 'real' faces of CGI characters - description in comments
So, I watched this Corridor Crew video on generating realistic faces from CG characters, and I wanted to try it out on the RDR2 models. The github link for the original work is here. If you guys are interested I can generate the faces of more characters from RDR2 and RDR1. I can even try some from RD Revolver.
-
AI Generated Art Scene Explodes as Hackers Create Groundbreaking New Tools - New AI tools CLIP+VQ-GAN can create impressive works of art based on just a few words of input.
Combining these methods with CLIP allows you to generate images based on text. This one uses a face generator. https://github.com/orpatashnik/StyleCLIP
- [D] How to save latent code edited from StyleClip.
clip-glass
-
test
(Added Feb. 5, 2021) CLIP-GLaSS.ipynb - Colaboratory by Galatolo. Uses BigGAN (default) or StyleGAN to generate images. The GPT2 config is for image-to-text, not text-to-image. GitHub.
-
Image to text models
After a cursory search I found CLIP-GLaSS and CLIP-cap. I've used CLIP-GLaSS in a previous experiment, but found the captions for digital/CG images quite underwhelming. This is understandable since this is not what the model was trained on, but still I'd like to use a better model.
-
[R] end-to-end image captioning
CLIP-GLaSS
- What CLIP-GLaSS thinks Ancient Egyptian computers would look like
-
Texttoimage 3 Images For Text Photo Of Donald
The images were generated using this notebook.
- CLIP-GLaSS prompt: "Screenshot of a video game from the 1930s"
-
[P] List of sites/programs/projects that use OpenAI's CLIP neural network for steering image/video creation to match a text description
The CLIP-GLaSS project has image-to-text functionality (I haven't tried it.)
-
For educational purposes: Text-to-image (3 runs with no cherry-picking, 6 images each) for text "Photo of a Lamborghini painted purple and red" generated using CLIP-GLaSS. config=StyleGAN2_car_d. save_each=50. generations=1000
Link to notebook.
-
Sharing CLIP magic based on OpenAI's blog post via a bit more accessible YT medium. Lmk what u think 🙈 ❤️
CLIP-GLaSS
-
[R] [P] Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search. Link to code and Google Colab notebook for project CLIP-GLaSS is in a comment.
Github for CLIP-GLaSS is here.
What are some alternatives?
encoder4editing - Official implementation of "Designing an Encoder for StyleGAN Image Manipulation" (SIGGRAPH 2021) https://arxiv.org/abs/2102.02766
a-PyTorch-Tutorial-to-Image-Captioning - Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning
compare_gan - Compare GAN code.
meshed-memory-transformer - Meshed-Memory Transformer for Image Captioning. CVPR 2020
NVAE - The Official PyTorch Implementation of "NVAE: A Deep Hierarchical Variational Autoencoder" (NeurIPS 2020 spotlight paper)
deep-daze - Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun
stylegan2-pytorch - Simplest working implementation of Stylegan2, state of the art generative adversarial network, in Pytorch. Enabling everyone to experience disentanglement
aphantasia - CLIP + FFT/DWT/RGB = text to image/video
pixel2style2pixel - Official Implementation for "Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation" (CVPR 2021) presenting the pixel2style2pixel (pSp) framework
stylized-neural-painting - Official Pytorch implementation of the preprint paper "Stylized Neural Painting", in CVPR 2021.
alias-free-gan - Alias-Free GAN project website and code
StyleCLIP - Using CLIP and StyleGAN to generate faces from prompts.