Show HN: Offline sketch to image geneartor in a whiteboard

• stable-diffusion.cpp

  11 2,806 8.7 C++

  Stable Diffusion in pure C/C++

TLDR: Miniconda + Diffusers + Electron + Excalidraw

One of the most important aspects of creating this tool was finding a stable diffusion locally inference solution. There are many solutions available, such as https://github.com/leejet/stable-diffusion.cpp and https://github.com/apple/ml-stable-diffusion. I tested the C++ version, but the inference speed was very slow, and Metal GPU support still had issues (you can find relevant issues in their repo). Ultimately, I decided to use python to run it because PyTorch is mature and MPS support is well-established. And I chose Miniconda, because it can create a small, isolated Python environment to run the program.

The AI model should run in the background so that we can continuously produce images while you draw. We need to find an RPC method to enable communication between the Python process and Electron's Node.js process. The easiest way is to run a Python HTTP server, but the memory usage is too high. We need a more lightweight solution, so I used xmlrpc for memory efficiency, although there might be better alternatives that I'm unaware of. The AI inference part is handled by diffusers, which is great, but I had to apply some custom patches to make it work in this situation. This can be a bit challenging if you're not familiar with Python.

For the frontend, I initially used a low-level canvas library and tried to implement a drawing pad from scratch. However, it had too many details, so I chose a more mature option: Excalidraw. It's fantastic, with the only shortcoming being limited API support.

Finally, I combined all these technologies in Electron, ensuring they work smoothly on both the main process and the renderer process.

Ok, Is DrawingPics free to use?

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• ml-stable-diffusion

  46 16,379 7.2 Python

  Stable Diffusion with Core ML on Apple Silicon

  

TLDR: Miniconda + Diffusers + Electron + Excalidraw

One of the most important aspects of creating this tool was finding a stable diffusion locally inference solution. There are many solutions available, such as https://github.com/leejet/stable-diffusion.cpp and https://github.com/apple/ml-stable-diffusion. I tested the C++ version, but the inference speed was very slow, and Metal GPU support still had issues (you can find relevant issues in their repo). Ultimately, I decided to use python to run it because PyTorch is mature and MPS support is well-established. And I chose Miniconda, because it can create a small, isolated Python environment to run the program.

The AI model should run in the background so that we can continuously produce images while you draw. We need to find an RPC method to enable communication between the Python process and Electron's Node.js process. The easiest way is to run a Python HTTP server, but the memory usage is too high. We need a more lightweight solution, so I used xmlrpc for memory efficiency, although there might be better alternatives that I'm unaware of. The AI inference part is handled by diffusers, which is great, but I had to apply some custom patches to make it work in this situation. This can be a bit challenging if you're not familiar with Python.

For the frontend, I initially used a low-level canvas library and tried to implement a drawing pad from scratch. However, it had too many details, so I chose a more mature option: Excalidraw. It's fantastic, with the only shortcoming being limited API support.

Finally, I combined all these technologies in Electron, ensuring they work smoothly on both the main process and the renderer process.

Ok, Is DrawingPics free to use?

• krita-ai-diffusion

  14 5,245 9.7 Python

  Streamlined interface for generating images with AI in Krita. Inpaint and outpaint with optional text prompt, no tweaking required.

I here a lot of people really like Stable Diffusion for Krita (https://github.com/Acly/krita-ai-diffusion). It's pretty easy to set up, powerful, local, and totally free.

Suggest a related project