clip-glass
Story2Hallucination
clip-glass | Story2Hallucination | |
---|---|---|
13 | 13 | |
177 | 146 | |
- | -0.7% | |
0.0 | 0.0 | |
over 2 years ago | about 3 years ago | |
Python | Jupyter Notebook | |
GNU General Public License v3.0 only | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
clip-glass
-
test
(Added Feb. 5, 2021) CLIP-GLaSS.ipynb - Colaboratory by Galatolo. Uses BigGAN (default) or StyleGAN to generate images. The GPT2 config is for image-to-text, not text-to-image. GitHub.
-
Image to text models
After a cursory search I found CLIP-GLaSS and CLIP-cap. I've used CLIP-GLaSS in a previous experiment, but found the captions for digital/CG images quite underwhelming. This is understandable since this is not what the model was trained on, but still I'd like to use a better model.
-
[R] end-to-end image captioning
CLIP-GLaSS
- What CLIP-GLaSS thinks Ancient Egyptian computers would look like
-
Texttoimage 3 Images For Text Photo Of Donald
The images were generated using this notebook.
- CLIP-GLaSS prompt: "Screenshot of a video game from the 1930s"
-
[P] List of sites/programs/projects that use OpenAI's CLIP neural network for steering image/video creation to match a text description
The CLIP-GLaSS project has image-to-text functionality (I haven't tried it.)
-
For educational purposes: Text-to-image (3 runs with no cherry-picking, 6 images each) for text "Photo of a Lamborghini painted purple and red" generated using CLIP-GLaSS. config=StyleGAN2_car_d. save_each=50. generations=1000
Link to notebook.
-
Sharing CLIP magic based on OpenAI's blog post via a bit more accessible YT medium. Lmk what u think 🙈 ❤️
CLIP-GLaSS
-
[R] [P] Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search. Link to code and Google Colab notebook for project CLIP-GLaSS is in a comment.
Github for CLIP-GLaSS is here.
Story2Hallucination
-
test
(Added Feb. 5, 2021) Story2Hallucination.ipynb - Colaboratory by bonkerfield. Uses BigGAN to generate images/videos. GitHub.
- Skrev in "Stockholm" i en AI-generator. Fick fram denna. Känns rimligt ändå
-
Some AI tools I've picked up, and tips.
Third which is pretty new to me is the Story2Hallucination. Which takes text, (say your story) and uses Google Deep Sleep to create a visual using AI to generate images for what the story is describing. And example can be found here.
-
Text to Image Generation
I ran some Lovecraft through Story2Hallucination[1] which uses Big Sleep to make videos from text.
The results were quite something - https://m.imgur.com/tfWLsSR
[1] https://github.com/lots-of-things/Story2Hallucination
-
[P] Visualizing evolution of Text-to-Image generation algorithms side by side by generating video from song-lyrics (X-LXMERT v/s AlpehImage/Dall-E)
Using Story2Hallucination: https://boredhumans.com/music_videos/Story2Hallucination_withwords.mp4 (made with https://github.com/lots-of-things/Story2Hallucination). The problem with it was that I had no easy way to match the timing of the lyrics on the screen with the real singing. So I then made a new version at https://boredhumans.com/music_videos/Story2Hallucination_nowords.mp4 where I edited the code so it did not show the words at all. But it is somewhat boring to watch.
-
Story2Hallucination renders a world of dragons from an AI dungeon game
Here you go: https://github.com/lots-of-things/Story2Hallucination/
-
[D] Will machine learning enable a single person to make a blockbuster movie like Avengers: endgame within 6 months?
This is basic attempt at that: https://github.com/lots-of-things/Story2Hallucination . It turns the text you write into a dream-like series of images converted into an animated GIF. It does not look real, but it is a start.
-
Story2Hallucination render of my latest meme (check comments for links)
Story2Hallucination github by u/bonkerfield : GITHUB LINK
-
Story2Hallucination on another AI Dungeon game and now I have a script to visualize to GIF while playing at the same time.
I've added a slightly simpler notebook to Story2Hallucination that can render GIFs on Google Colab. Note that the story text has to be fairly short or it will make a gigantic unrenderable GIF.
-
Short video generated by Story2Hallucination
Credit to https://github.com/lots-of-things/Story2Hallucination/ and onwards - I converted the script to run in regular python, and on windows through WSL+CUDA (Though, the windows tweaks seem to have caused other issues, Will probably have to roll back and dualboot on this device to do more)
What are some alternatives?
a-PyTorch-Tutorial-to-Image-Captioning - Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning
deep-daze - Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun
meshed-memory-transformer - Meshed-Memory Transformer for Image Captioning. CVPR 2020
big-sleep - A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun
deep-music-visualizer - The Deep Visualizer uses BigGAN (Brock et al., 2018) to visualize music.
aphantasia - CLIP + FFT/DWT/RGB = text to image/video
DALLE-pytorch - Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
stylized-neural-painting - Official Pytorch implementation of the preprint paper "Stylized Neural Painting", in CVPR 2021.
StyleCLIP - Official Implementation for "StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery" (ICCV 2021 Oral)
StyleCLIP - Using CLIP and StyleGAN to generate faces from prompts.