DALLE-pytorch VS pyp

It all started originally on lucidrains/dalle-pytorch in the months following the release of DALL-E (1). The group started as `dalle-pytorch-replicate` but was never officially "blessed" by Phil Wang who seems to enjoy being a free agent (can't blame him).

https://github.com/lucidrains/DALLE-pytorch/issues/116 is where the discord got kicked off originally. There's a lot of other interactions between us in the github there. You should be able to find when Phil was approached by Jenia Jitsev, Jan Ebert, and Mehdi Cherti (all starting LAION members) who graciously offered the chance to replicate the DALL-E paper using their available compute at the JUWELS and JUWELS Booster HPC system. This all predates Emad's arrival. I believe he showed up around the time guided diffusion and GLIDE, but it may have been a bit earlier.

Data work originally focused on amassing several of the bigger datasets of the time. Getting CC12M downloaded and trained on was something of an early milestone (robvanvolt's work). A lot of early work was like that though, shuffling through CC12M, COCO, etc. with the dalle-pytorch codebase until we got an avocado armchair.

Christophe Schumann was an early contributor as well and great at organizing and rallying. He focused a lot on the early data scraping work for what would become the "LAION5B" dataset. I don't want to credit him with the coding and I'm ashamed to admit I can't recall who did much of the work there - but a distributed scraping program was developed (the name was something@home... not scraping@home?).

The discord link on Phil Wang's readme at dalle-pytorch got a lot of traffic and a lot of people who wanted to pitch in with the scraping effort.

Eventually a lot of people from Eleuther and many other teams mingled with us, some sort of non-profit org was created in Germany I believe for legal purposes. The dataset continued to grow and the group moved from training DALLE's to finetuning diffusion models.

The `CompVis` team were great inspiration at the time and much of their work on VQGAN and then latent diffusion models basically kept us motivated. As I mentioned a personal motivation was Katherine Crowson's work on a variety of things like CLIP-guided vqgan, diffusion, etc.

I believe Emad Mostaque showed up around the time GLIDE was coming out? I want to say he donated money for scrapers to be run on AWS to speed up data collection. I was largely hands off for much of the data scraping process and mostly enjoyed training new models on data we had.

As with any online community things got pretty ill-defined, roles changed over, volunteers came/went, etc. I would hardly call this definitive and that's at least partially the reason it's hard to trace as an outsider. That much of the early history is scattered about GitHub issues and PR's can't have helped though.

Here you go: https://github.com/lucidrains/DALLE-pytorch

Here are some other implementations of Dalle clones in Pytorch by various authors in the ML and DL community: https://github.com/lucidrains/DALLE-pytorch

If in general DDPM > GAN > VAE, why do transformer image generators all use VQVAE to decode images? Wouldn't it be better to use a diffusion model? I was wondering about this and started experimenting with different ways to decode vector-quantized embeddings with a diffusion model - see discussion here After a lot of trial and error I got something that works pretty well.

- "Discrete VAE", used as the backbone for OpenAI's DALL-E, reimplimented here (and other places) https://github.com/lucidrains/DALLE-pytorch (code for training a discrete VAE)

Here's the DALLE-pytorch git repo.

I am very excited, when AI Dungeon was released and seeing them filtering stuff, I thought that one day there will be an open source version of this without filters, the same goes for any future open sourced GPT-X. Now if we can get to train an open source DALL-E too and integrate it on NovelAI. Wouldn't that be even more awesome?

Vielleicht bei https://github.com/lucidrains/DALLE-pytorch und https://github.com/kobiso/DALLE-reproduction

> I too can never remember jq syntax when I need to. I usually just end up writing a Python script

Same here! That's why for small things I made pyxargs [1] to use python in the shell. In another thread I also just learned of pyp [2] which I haven't tried yet but looks like it'd be even better for this use case.

[1] https://github.com/elesiuta/pyxargs

[2] https://github.com/hauntsaninja/pyp

I have bookmarked/tried so many Python/Shell mashups over the years.

IMHO the following is about the only one that's tasteful and not going off the deep end: https://github.com/hauntsaninja/pyp

I wrote a similar tool a while back that lets you create your own "magic" variables. I use `f` all the time! https://github.com/hauntsaninja/pyp#pyp-lets-you-configure-y...

Thank you for the extensive and thoughtful comment! This does help clarify your approach quite considerably. I wonder, since you are hoping to attract collaborators, whether there is some kind of formal spec for the language somewhere? For example, you mentioned parallel efforts: suppose I wanted to write a port to pure C; is there any way, short of reading every one of your posts and trying to contain the whole language in my head at once, for me to know exactly what I need to implement?

Something I've been trying to figure out: what is the exact relationship at present between OSH and Oil? When you say "OSH" do you mean the language, or the shell itself "oil shell"? If Oil is not something I can download, why exactly does that `const v = max(1, 2)` statement work in osh? It's clearly not just a Bash implementation, it's got other features. Is that a subset of Oil's features? Which subset?

Since you're also interested in other shells, you might have a look at pyp [1]. It captures a lot of the way I personally would like to use some future shell. If the features of pyp were integrated into the shell itself, you wouldn't need an external command, you could just (for example) pipe the output of one program into a python-like statement that mangles the incoming strings in some way, and pipe that out to some xargs-like program to use in a subshell. (The fact that you apparently can't use the pipe in what Xonsh calls "Python mode" is for me the central limiting feature of that shell.)

[1] https://github.com/hauntsaninja/pyp

9. Pyp