swift-diffusion VS fickling

I am going to put model related code we use in a public repo soon (it is very similar to https://github.com/liuliu/swift-diffusion but in NHWC format). ANE will be around 25s if it runs. DT's default only uses GPUs and 35s is on GPU (yes, like you said, upscaling would take extra 10s).

The text encoder uses a new vocabulary set, make sure you copied them from open_clip repo: https://github.com/mlfoundations/open_clip (I have these also available at: https://github.com/liuliu/swift-diffusion/tree/liu/unet/examples/open_clip

Should be able too, if there is a need. I am more interested to support training hypernetwork from the app directly. The conversion script itself is open-source (https://github.com/liuliu/swift-diffusion/blob/main/examples/unet/main.swift), but not polished, and because Apple doesn't allow you to run Python on device, so I cannot make it as easy as typing a URL and get done. Need to figure out what the UX looks like without me providing a networked services ...

Hi, this is the first app in a while (probably 10 years) that I submitted to AppStore. I've done this app in 3 weeks, so there are a lot to be polished. The technology that enables this I discussed in depth in an accompanied blog post: https://liuliu.me/eyes/stretch-iphone-to-its-limit-a-2gib-mo...

Some parts of it (or major parts) is also available at https://github.com/liuliu/swift-diffusion. I plan to port more stuff back to swift-diffusion and make a CLI tool out of it (it is a bit more work than the app because I need to consider CUDA compatibility there).

AMA!

You're right! You should probably use Trail of Bits Fickling tool to investigate. https://github.com/trailofbits/fickling

I tested the Anything V3 pruned from Hugging Face, and indeed nothing funny in its pickle. I used the Fickling library to decompile it. I do not use Windows so my interests in .ckpt security are largely related to Pickle exploits— which could extract malicious code from a data file and then do something with it, but the data files themselves are not executed. I will edit this comment with lines referencing that data file.

I've been using Diffusion Bee on my Mac, and it's just gained the ability to import models (which it converts), but it is unpickling to do so— but barely. It unpickles, figures out what sort of data is in every data file and then computes what it wants from them on its own. I would love it to not use unpickling at all, so my intention is if I can figure it out, to write a script to decode the pickle file (with Fickling or otherwise) and then just do the weight calculation/assignment.

Something you won't gather from skim-reading the headline is that this is that the author has also created a tool, Fickling: https://github.com/trailofbits/fickling - to aid in playing around with pickle files.

From the article: [Fickling] can help you reverse engineer, test, and even create malicious pickle files.