web-stable-diffusion VS gpuweb

Yup, here's their web stable diffusion repo: https://github.com/mlc-ai/web-stable-diffusion

The input is a model (weights + runtime lib) compiled via the mlc-llm project: https://mlc.ai/mlc-llm/docs/compilation/compile_models.html

The MLC team got that working back in March: https://github.com/mlc-ai/web-stable-diffusion

Even more impressively, they followed up with support for several Large Language Models: https://webllm.mlc.ai/

[https://github.com/mlc-ai/web-stable-diffusion

You can run it directly in the browser with WebGPU, https://mlc.ai/web-stable-diffusion/

Using the amazing work of https://mlc.ai/web-stable-diffusion/ I've got the code moved into a Web Worker and running fully local client side. It does require 2GB's of model files be downloaded (automatically), and takes a few minutes for the first load, but it works and once it's going it only takes 20s to make a 512x512 image.

The Apache TVM machine learning compiler has a WASM and WebGPU backend, and can import from most DNN frameworks. Here's a project running Stable Diffusion with webgpu and TVM [1].

Questions exist around post-and-pre-processing code in folks' Python stacks, with e.g. NumPy and opencv. There's some NumPy to JS transpilers out there, but those aren't feature complete or fully integrated.

[1] https://github.com/mlc-ai/web-stable-diffusion

Works for me.

WebGPU support is behind a couple flags on Linux: https://github.com/gpuweb/gpuweb/wiki/Implementation-Status

People keep spreading this incredibly misleading statement, and yours is even more misleading (suggesting Apple opposed a 'GPU WASM')

By all accounts, Apple's /only/ stance was that if WebGPU used SPIR-V it would be a non-starter for them, due to ongoing legal issues between Apple and Khronos.

Apple actually proposed WebHLSL in collaboration with Microsoft, to have HLSL be the standard.

Mozilla employee's stance[0] was that SPIRV was too low level, did not fit with the goals of WebGPU portability and security, and expressed concern that Khronos may add functionality to SPIRV they cannot support in WebGPU like raytracing instructions .. 'So we'd always be on the verge of forking SPIR-V in some way.'

It was also noted by many people that even if a bytecode format was used, it would still have to be translated to the target (HLSL/DXIL, MSL, etc.) in almost the same way a text format would.

Nobody proposed a 'GPU WASM equivalent' or an alternative bytecode format.

The hard truth is that shader compilation is a fucking nightmare, people do not realize how bad it is across the different native APIs. SPIR-V is good, but it doesn't solve that - and presents other challenges if you are a web browser API. Vulkan and SPIRV are not the golden goose many make them out to be.

[0] https://github.com/gpuweb/gpuweb/issues/847#issuecomment-642...

Yes it is still a bit new. WebGPU is not finished and is still being worked on: https://webgpu.io/

WebGPU currently doesn't support the "bindless" resource access model (see: https://github.com/gpuweb/gpuweb/issues/380).

The "max number of sampled texture per shader stage" is a runtime device limit, and the minimal value for that seems to be 16. So texture atlasses are still a thing in WebGPU.

There's even an implementation under discussion to have the dp4a instruction added to WebGPU (https://github.com/gpuweb/gpuweb/issues/2677)