SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 text-to-image Open-Source Projects
-
DALLE2-pytorch
Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Dreambooth-Stable-Diffusion
Implementation of Dreambooth (https://arxiv.org/abs/2208.12242) with Stable Diffusion
-
DALLE-pytorch
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
-
deep-daze
Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun
-
Awesome-Prompt-Engineering
This repository contains a hand-curated resources for Prompt Engineering with a focus on Generative Pre-trained Transformer (GPT), ChatGPT, PaLM etc
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
dalle-playground
A playground to generate images from any text prompt using Stable Diffusion (past: using DALL-E Mini)
-
-
big-sleep
A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun
-
awesome-generative-ai
A curated list of Generative AI tools, works, models, and references (by filipecalegario)
-
-
CogView
Text-to-Image generation. The repo for NeurIPS 2021 paper "CogView: Mastering Text-to-Image Generation via Transformers".
-
TokenFlow
Official Pytorch Implementation for "TokenFlow: Consistent Diffusion Features for Consistent Video Editing" presenting "TokenFlow" (ICLR 2024)
-
text2room
Text2Room generates textured 3D meshes from a given text prompt using 2D text-to-image models (ICCV2023).
-
CogView2
official code repo for paper "CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers"
-
MultiDiffusion
Official Pytorch Implementation for "MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation" presenting "MultiDiffusion" (ICML 2023)
-
muse-maskgit-pytorch
Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
I was showing people Dalle2 last year and telling them how much of an impact an open source solution was going to have on, well, everything to do with art and design. (At the time Stable Diffusion had not released, not even the leak, and all hopes was on https://github.com/lucidrains/DALLE2-pytorch)
Project mention: Google's StyleDrop can transfer style from a single image | /r/StableDiffusion | 2023-06-03If google doesnt, someone like lucidrains probably would implement it, just like he did for imagen and muse.
It all started originally on lucidrains/dalle-pytorch in the months following the release of DALL-E (1). The group started as `dalle-pytorch-replicate` but was never officially "blessed" by Phil Wang who seems to enjoy being a free agent (can't blame him).
https://github.com/lucidrains/DALLE-pytorch/issues/116 is where the discord got kicked off originally. There's a lot of other interactions between us in the github there. You should be able to find when Phil was approached by Jenia Jitsev, Jan Ebert, and Mehdi Cherti (all starting LAION members) who graciously offered the chance to replicate the DALL-E paper using their available compute at the JUWELS and JUWELS Booster HPC system. This all predates Emad's arrival. I believe he showed up around the time guided diffusion and GLIDE, but it may have been a bit earlier.
Data work originally focused on amassing several of the bigger datasets of the time. Getting CC12M downloaded and trained on was something of an early milestone (robvanvolt's work). A lot of early work was like that though, shuffling through CC12M, COCO, etc. with the dalle-pytorch codebase until we got an avocado armchair.
Christophe Schumann was an early contributor as well and great at organizing and rallying. He focused a lot on the early data scraping work for what would become the "LAION5B" dataset. I don't want to credit him with the coding and I'm ashamed to admit I can't recall who did much of the work there - but a distributed scraping program was developed (the name was something@home... not scraping@home?).
The discord link on Phil Wang's readme at dalle-pytorch got a lot of traffic and a lot of people who wanted to pitch in with the scraping effort.
Eventually a lot of people from Eleuther and many other teams mingled with us, some sort of non-profit org was created in Germany I believe for legal purposes. The dataset continued to grow and the group moved from training DALLE's to finetuning diffusion models.
The `CompVis` team were great inspiration at the time and much of their work on VQGAN and then latent diffusion models basically kept us motivated. As I mentioned a personal motivation was Katherine Crowson's work on a variety of things like CLIP-guided vqgan, diffusion, etc.
I believe Emad Mostaque showed up around the time GLIDE was coming out? I want to say he donated money for scrapers to be run on AWS to speed up data collection. I was largely hands off for much of the data scraping process and mostly enjoyed training new models on data we had.
As with any online community things got pretty ill-defined, roles changed over, volunteers came/went, etc. I would hardly call this definitive and that's at least partially the reason it's hard to trace as an outsider. That much of the early history is scattered about GitHub issues and PR's can't have helped though.
Yes, there are a lot of different resources online, especially for generative AI. The Awesome Prompt Engineering github is probably a good place to start https://github.com/promptslab/Awesome-Prompt-Engineering. If you're focusing directly on OpenAI's models then the OpenAI Prompt Engineering Guide would be my recommendation https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api.
Built on dalle-playground because it is simple and I like it.
Project mention: New Kandinsky 2.2 was released. Now with controlnets and code for lora fine-tuning. | /r/StableDiffusion | 2023-07-13
Project mention: 📚 Tutorials & 🎨 AI Art Generation Tool List Mega Thread | /r/AI_Aesthetics | 2023-07-26VQGAN-CLIP
Project mention: Generative AI – A curated list of Generative AI tools, works, models | news.ycombinator.com | 2023-07-14
Code: https://github.com/omerbt/TokenFlow
Project mention: Opendream: A Non-Destructive UI for Stable Diffusion | news.ycombinator.com | 2023-08-15For composing this approach works pretty well
https://multidiffusion.github.io/
Project mention: Google's StyleDrop can transfer style from a single image | /r/StableDiffusion | 2023-06-03If google doesnt, someone like lucidrains probably would implement it, just like he did for imagen and muse.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
text-to-image related posts
-
Are you looking for free open source alternative to Midjourney and Bing images?
-
100% free Midjourney alternative. Plant trees as you generated realistic images
-
Are you looking for a green yet free Chatgpt4 Alternative?
-
A green tree planting alternative for chatgpt4, Gemini-pro and Midjourney
-
Green Chatgt4 Alternative That Plants Trees for Every Chat
-
Transforming the Future of Generative AI with Open Access and Global Impact
-
📚 Tutorials & 🎨 AI Art Generation Tool List Mega Thread
-
A note from our sponsor - SaaSHub
www.saashub.com | 3 May 2024
Index
What are some of the best open-source text-to-image projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | DALLE2-pytorch | 10,826 |
| 2 | imagen-pytorch | 7,787 |
| 3 | Dreambooth-Stable-Diffusion | 7,383 |
| 4 | DALLE-pytorch | 5,493 |
| 5 | deep-daze | 4,379 |
| 6 | min-dalle | 3,474 |
| 7 | Awesome-Prompt-Engineering | 3,212 |
| 8 | dalle-playground | 2,762 |
| 9 | Kandinsky-2 | 2,695 |
| 10 | Diffusion-Models-Papers-Survey-Taxonomy | 2,670 |
| 11 | VQGAN-CLIP | 2,563 |
| 12 | big-sleep | 2,559 |
| 13 | carefree-creator | 2,117 |
| 14 | awesome-generative-ai | 1,987 |
| 15 | Awesome-Text-to-Image | 1,878 |
| 16 | ru-dalle | 1,639 |
| 17 | CogView | 1,602 |
| 18 | TokenFlow | 1,462 |
| 19 | Radiata | 983 |
| 20 | text2room | 972 |
| 21 | CogView2 | 929 |
| 22 | MultiDiffusion | 906 |
| 23 | muse-maskgit-pytorch | 816 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com