Top 23 text2image Open-Source Projects

• stable-diffusion-webui

  2,808 133,415 9.9 Python

  Stable Diffusion web UI

Project mention: Show HN: I made an app to use local AI as daily driver | news.ycombinator.com | 2024-02-27

* LLaVA model: I'll add more documentation. You are right Llava could not generate images. For image generation I don't have immediate plans, but checkout these projects for local image generation.

- https://diffusionbee.com/

- https://github.com/comfyanonymous/ComfyUI

- https://github.com/AUTOMATIC1111/stable-diffusion-webui

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• diffusers

  268 23,519 9.9 Python

  🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.

  

Project mention: Diffusion Models | news.ycombinator.com | 2024-05-25

• PaddleHub

  9 12,576 1.9 Python

  Awesome pre-trained models toolkit based on PaddlePaddle. (400+ models including Image, Text, Audio, Video and Cross-Modal with Easy Inference & Serving)

  

• mmagic

  5 6,677 8.7 Jupyter Notebook

  OpenMMLab Multimodal Advanced, Generative, and Intelligent Creation Toolbox. Unlock the magic 🪄: Generative-AI (AIGC), easy-to-use APIs, awsome model zoo, diffusion models, for text-to-image generation, image/video restoration/enhancement, etc.

  

• StabilityMatrix

  11 3,119 9.9 C#

  Multi-Platform Package Manager for Stable Diffusion

  

Project mention: Meta's new AI image generator was trained on 1.1B Instagram and FB photos | news.ycombinator.com | 2023-12-07

On Windows, StabilityMatrix (https://github.com/LykosAI/StabilityMatrix) is a very easy way to get any (or all) of those wrappers installed without conflicts.

• stable-diffusion.cpp

  11 2,806 8.7 C++

  Stable Diffusion in pure C/C++

Project mention: Show HN: Offline sketch to image geneartor in a whiteboard | news.ycombinator.com | 2024-05-26

TLDR: Miniconda + Diffusers + Electron + Excalidraw

One of the most important aspects of creating this tool was finding a stable diffusion locally inference solution. There are many solutions available, such as https://github.com/leejet/stable-diffusion.cpp and https://github.com/apple/ml-stable-diffusion. I tested the C++ version, but the inference speed was very slow, and Metal GPU support still had issues (you can find relevant issues in their repo). Ultimately, I decided to use python to run it because PyTorch is mature and MPS support is well-established. And I chose Miniconda, because it can create a small, isolated Python environment to run the program.

The AI model should run in the background so that we can continuously produce images while you draw. We need to find an RPC method to enable communication between the Python process and Electron's Node.js process. The easiest way is to run a Python HTTP server, but the memory usage is too high. We need a more lightweight solution, so I used xmlrpc for memory efficiency, although there might be better alternatives that I'm unaware of. The AI inference part is handled by diffusers, which is great, but I had to apply some custom patches to make it work in this situation. This can be a bit challenging if you're not familiar with Python.

For the frontend, I initially used a low-level canvas library and tried to implement a drawing pad from scratch. However, it had too many details, so I chose a more mature option: Excalidraw. It's fantastic, with the only shortcoming being limited API support.

Finally, I combined all these technologies in Electron, ensuring they work smoothly on both the main process and the renderer process.

Ok, Is DrawingPics free to use?

• Kandinsky-2

  15 2,710 5.7 Jupyter Notebook

  Kandinsky 2 — multilingual text2image latent diffusion model

  

Project mention: New Kandinsky 2.2 was released. Now with controlnets and code for lora fine-tuning. | /r/StableDiffusion | 2023-07-13

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• VQGAN-CLIP

  67 2,563 0.0 Python

  Just playing with getting VQGAN+CLIP running locally, rather than having to use colab.

Project mention: 📚 Tutorials & 🎨 AI Art Generation Tool List Mega Thread | /r/AI_Aesthetics | 2023-07-26

VQGAN-CLIP

• voltaML-fast-stable-diffusion

  14 957 9.7 Python

  Beautiful and Easy to use Stable Diffusion WebUI

  

Project mention: Show HN: Shortbread, a web app that helps you create AI comics in minutes | news.ycombinator.com | 2023-10-06

Also, VoltaML has a good reference GPU AITemplate SD 1.5 implementation:

https://github.com/VoltaML/voltaML-fast-stable-diffusion/tre...

The speed jump is massive on my desktop GPU, probably even more dramatic on cloud hardware, and it may support some things (weight swapping/lora swapping/resolution changing) better than JAX.

• Pallaidium

  8 837 9.1 Python

  Generative AI for the Blender VSE: Text, video or image to video, image and audio in Blender Video Sequence Editor.

Project mention: Würstchen is here! A game changing, fastest high-resolution image generation model | /r/StableDiffusion | 2023-09-16

Get Pallaidium for free here: https://github.com/tin2tin/Pallaidium

• unprompted

  47 757 8.7 Python

  Templating language written for Stable Diffusion workflows. Available as an extension for the Automatic1111 WebUI.

Project mention: Unprompted v10 Released: New faceswap features, GPEN support, Civitai shortcode and more! 😊 | /r/StableDiffusion | 2023-10-13

I'm pleased to announce the release of Unprompted v10.0.0, the Swiss Army knife extension for A1111. This is a major update that brings a number of new features and improvements, including:

• stable-diffusion-aesthetic-gradients

  14 703 10.0 Jupyter Notebook

  Personalization for Stable Diffusion via Aesthetic Gradients 🎨

• Text2Light

  4 541 1.8 Jupyter Notebook

  [SIGGRAPH Asia 2022] Text2Light: Zero-Shot Text-Driven HDR Panorama Generation

• CLIP-Guided-Diffusion

  4 379 0.0 Python

  Just playing with getting CLIP Guided Diffusion running locally, rather than having to use colab. (by nerdyrodent)

• diffusers-interpret

  15 259 10.0 Jupyter Notebook

  Diffusers-Interpret 🤗🧨🕵️‍♀️: Model explainability for 🤗 Diffusers. Get explanations for your generated images.

• tiny-dream

  6 245 7.1 C

  Tiny Dream - An embedded, Header Only, Stable Diffusion C++ implementation

Project mention: Tiny Dream – An Embedded, Header Only, Stable Diffusion C++ Implementation | news.ycombinator.com | 2023-09-26

• onnx-web

  8 181 9.6 Python

  web UI for GPU-accelerated ONNX pipelines like Stable Diffusion, even on Windows and AMD

• StableDiffusionUI

  9 126 8.6 Python

  Stable Diffusion UI: Diffusers (CUDA/ONNX)

• phrame

  11 119 7.3 TypeScript

  AI-powered digital picture frame. Generate captivating and unique art from spoken conversations.

Project mention: What are you building with OpenAI? | /r/OpenAIDev | 2023-07-03

I built and open-sourced Phrame, a digital picture frame that uses OpenAI's ChatGPT and DALL·E 2 to generate AI art.

• yasd-discord-bot

  14 112 10.0 Python

  Yet Another Stable Diffusion Discord Bot

• vqgan-clip-app

  3 100 0.0 Python

  Local image generation using VQGAN-CLIP or CLIP guided diffusion

• AIConnectors

  8 94 4.1 C#

  Unity C# API connections to StableDiffusion (Automatic1111, Stability.ai SDXL, Replicate.com), Dall-E, ChatGPT/ GPT-4, and possibly others in the future.

• infinite-zoom-stable-diffusion

  8 82 5.8 Jupyter Notebook

  resources for creating Ininite zoom video using Stable Diffiusion, you can use multiple prompts and it is easy to use.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

text2image related posts

• 📚 Tutorials & 🎨 AI Art Generation Tool List Mega Thread

  1 project | /r/AI_Aesthetics | 26 Jul 2023

• I hope that by A1111 is everything fine. Best wishes for great man! In mean time lets, develope Vlad fork

  7 projects | /r/StableDiffusion | 23 Apr 2023

• another UI for Stable Diffusion for Windows and AMD, now with LoRA and Textual Inversions

  9 projects | /r/StableDiffusion | 28 Mar 2023

• Which is your favorite text to image model overall?

  6 projects | /r/StableDiffusion | 23 Sep 2022

• Stable Diffusion Announcement

  3 projects | news.ycombinator.com | 10 Aug 2022

• Medieval Noir - VQGAN-CLIP - COCO Checkpoint

  1 project | /r/MediaSynthesis | 10 Jul 2022

• Once have access, do you run it on your computer or over the internet on Open-AI's computers?

  3 projects | /r/dalle2 | 6 Jun 2022
• A note from our sponsor - InfluxDB
  www.influxdata.com | 15 Jun 2024

  Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

Sponsored

Free Django app performance insights with Scout Monitoring

Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

www.scoutapm.com