guided-diffusion VS pytorch-lightning

People who made these colabs better and better also the same people who are at Midjourney now. But the "mother" of it all, was Katherine Crowson. She made a fine tuned model that uses a 512x512 unconditional ImageNet diffusion model fine-tuned from OpenAI's 512x512 class-conditional ImageNet diffusion model (https://github.com/openai/guided-diffusion) together with CLIP (https://github.com/openai/CLIP) to connect text prompts with images. It uses a smaller secondary diffusion model trained by Katherine Crowson to remove noise from intermediate timesteps to prepare them for CLIP.

I am trying to use OpenAI's Guided Diffusion Github to train my own diffusion model. I thought to ask here to see if anyone had any experience with it as I've been having trouble training my own models on it. If anyone has any resources to point me towards it would be greatly appreciated!

Our codebase for the diffusion models builds heavily on OpenAI's ADM codebase , lucidrains, Stable Diffusion, Lightning and Hugging Face. Thanks for open-sourcing!

I am working with guided diffusion. I would like to reproduce the results of the repository for the 64->256 super resolution network. https://github.com/openai/guided-diffusion

this code is (mostly) just the original openai guided diffusion code: https://github.com/openai/guided-diffusion

Starting resource, as it is all done through this code (information on how to do it on Colab is out there) https://github.com/openai/guided-diffusion

Original notebook by Katherine Crowson (https://github.com/crowsonkb, https://twitter.com/RiversHaveWings). It uses either OpenAI's 256x256 unconditional ImageNet or Katherine Crowson's fine-tuned 512x512 diffusion model (https://github.com/openai/guided-diffusion), together with CLIP (https://github.com/openai/CLIP) to connect text prompts with images.

Code for https://arxiv.org/abs/2105.05233 found: https://github.com/openai/guided-diffusion

Paper, repo, Colab. It's really good.

It's very easy to get started, right in your Terminal, no fees! No credit card at all.

And there are cloud providers like https://replicate.com/ and https://lightning.ai/ that will let you use your LLM via an API key just like you did with OpenAI if you need that.

You don't need OpenAI - nobody does.

https://see.stanford.edu/Course/CS229 https://lightning.ai/ https://www.youtube.com/watch?v=00s9ireCnCw&t=57s https://towardsdatascience.com/

I've been wondering on what is the recommended method of saving logits/activations using PyTorch Lightning. I've looked at Callbacks, Loggers and ModelHooks but none of the use-cases seem to be for this kind of activity (even if I were to create my own custom variants of each utility). The ModelCheckpoint Callback in its utility makes me feel like custom Callbacks would be the way to go but I'm not quite sure. This closed GitHub issue does address my issue to some extent.

Our codebase for the diffusion models builds heavily on OpenAI's ADM codebase , lucidrains, Stable Diffusion, Lightning and Hugging Face. Thanks for open-sourcing!

For lightweight use, pytorch-lightning is too heavy, and its source code will be very difficult for beginners to read, at least for me.

For the A100 GPU, theoretical performance is the same for FP16/BF16 and both rely on the same number of bits, meaning memory should be the same. However since it's quite newly added to PyTorch, performance seems to still be dependent on underlying operators used (pytorch lightning debugging in progress here).