Show HN: Scribble Diffusion – Turn your sketch into a refined image using AI

• scribble-diffusion

  28 2,851 5.5 JavaScript

  Turn your rough sketch into a refined image using AI

  

Hey, Scribble Diffusion author here.

Thanks for your kinds words and feedback. I love seeing all these links to your scribbles.

I'm an engineer at Replicate, which is a place to run ML models in the cloud. We built Scribble Diffusion to demonstrate how to use Replicate. It's open source: https://github.com/replicate/scribble-diffusion

This is all built on ControlNet, a brilliant technique by Lyumin Zhang for controlling the outputs of diffusion models like Stable Diffusion. https://github.com/lllyasviel/ControlNet

Here's another example of someone doing something cool with ControlNet: https://twitter.com/dannypostmaa/status/1630442372206133248 -- it lets you pose a stick figure and generate DreamBooth images that match the pose.

I see a few bits of feedback here about issues with the Scribble Diffusion UI, and I'm tracking them on the GitHub repo. If you want to help out, please feel free to open an issue or pull request.

• sd-webui-controlnet

  247 15,859 9.7 Python

  WebUI extension for ControlNet

This looks like its just a quick front-end to the ControlNet scribble model (perhaps with a customized underlying model instead of base Stable Diffusion), with canned settings, presumable a canned negative prompt, and maybe some common stuff added beyond the user input to the positive prompt. Which is not to be dismissive, its a very nice demo of what SD+ControlNet scribble can do.

But for people who like it, the ControlNet scribble model (and the other ControlNet models, depth-map based, pose control, edge detection, etc.) [0] are supported in the ControlNet extension [1] to the A1111 Stable Diffusion Web UI [2], and probably similar extensions for other popular stable diffusion UIs. Should work in any current browser, and at least the A1111 UI, with ControlNet models, works on machines with as little as 4GB VRAM.

[0] home repo: https://huggingface.co/lllyasviel/ControlNet but for WebUI you probably want the ones linked from the readme of the WebUI ControlNet extensions [1]

[1] https://github.com/Mikubill/sd-webui-controlnet

[2] https://github.com/AUTOMATIC1111/stable-diffusion-webui

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• stable-diffusion-webui

  2,808 129,299 9.9 Python

  Stable Diffusion web UI

This looks like its just a quick front-end to the ControlNet scribble model (perhaps with a customized underlying model instead of base Stable Diffusion), with canned settings, presumable a canned negative prompt, and maybe some common stuff added beyond the user input to the positive prompt. Which is not to be dismissive, its a very nice demo of what SD+ControlNet scribble can do.

But for people who like it, the ControlNet scribble model (and the other ControlNet models, depth-map based, pose control, edge detection, etc.) [0] are supported in the ControlNet extension [1] to the A1111 Stable Diffusion Web UI [2], and probably similar extensions for other popular stable diffusion UIs. Should work in any current browser, and at least the A1111 UI, with ControlNet models, works on machines with as little as 4GB VRAM.

[0] home repo: https://huggingface.co/lllyasviel/ControlNet but for WebUI you probably want the ones linked from the readme of the WebUI ControlNet extensions [1]

[1] https://github.com/Mikubill/sd-webui-controlnet

[2] https://github.com/AUTOMATIC1111/stable-diffusion-webui

• react-sketch-canvas

  2 362 6.6 TypeScript

  Freehand vector drawing component for React using SVG as canvas 🖌️

Per github seems to be related to:

https://github.com/vinothpandian/react-sketch-canvas/pull/11...

• ControlNet

  127 27,776 4.1 Python

  Let us control diffusion models!

Hey, Scribble Diffusion author here.

Thanks for your kinds words and feedback. I love seeing all these links to your scribbles.

I'm an engineer at Replicate, which is a place to run ML models in the cloud. We built Scribble Diffusion to demonstrate how to use Replicate. It's open source: https://github.com/replicate/scribble-diffusion

This is all built on ControlNet, a brilliant technique by Lyumin Zhang for controlling the outputs of diffusion models like Stable Diffusion. https://github.com/lllyasviel/ControlNet

Here's another example of someone doing something cool with ControlNet: https://twitter.com/dannypostmaa/status/1630442372206133248 -- it lets you pose a stick figure and generate DreamBooth images that match the pose.

I see a few bits of feedback here about issues with the Scribble Diffusion UI, and I'm tracking them on the GitHub repo. If you want to help out, please feel free to open an issue or pull request.

• artbot-for-stable-diffusion

  85 157 9.4 TypeScript

  A front-end GUI for interacting with the AI Horde / Stable Diffusion distributed cluster

I recently added support for it on ArtBot[1] and it supports additional options like models, steps, control type, etc, as well drawing on a canvas[2] and generating input from that. It’s pretty wild!

Free and open source [3].

Bonus: it works on Firefox (unless you’re using private mode — because Firefox doesn’t make IndexedDb available to web apps in private mode and things end up breaking)

[1] https://tinybots.net/artbot/controlnet

[2] https://tinybots.net/artbot/draw

[3] https://github.com/daveschumaker/artbot-for-stable-diffusion

Suggest a related project