Run Your Own DALL·E Mini (Craiyon) Server on EC2

This page summarizes the projects mentioned and recommended in the original post on dev.to

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • dalle-mini

    DALL·E Mini - Generate images from a text prompt

    In case you’ve been under a rock for the last few months, DALL·E is an ML model from OpenAI that generates images from text prompts. DALL·E Mini (renamed to Craiyon) by Boris Dayma et al. is a less powerful but open version of DALL·E, and there’s a hosted version at craiyon.com for everyone to try.

  • meadowrun

    Discontinued Meadowrun makes it easy to run your code on the cloud

    If you’re anything like us, though, you’ll feel compelled to poke around the code and run the model yourself. We’ll do that in this article using Meadowrun, an open-source library that makes it easy to run Python code in the cloud. For ML models in particular, we just added a feature for requesting GPU machines in a recent release. We’ll also feed the images generated by DALL·E Mini into additional image processing models (GLID-3-xl and SwinIR) to improve the quality of our generated images. Along the way we’ll deal with the speedbumps that come up when running open-source ML models on EC2.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • glid-3-xl

    1.4B latent diffusion model fine tuning

    If you’re anything like us, though, you’ll feel compelled to poke around the code and run the model yourself. We’ll do that in this article using Meadowrun, an open-source library that makes it easy to run Python code in the cloud. For ML models in particular, we just added a feature for requesting GPU machines in a recent release. We’ll also feed the images generated by DALL·E Mini into additional image processing models (GLID-3-xl and SwinIR) to improve the quality of our generated images. Along the way we’ll deal with the speedbumps that come up when running open-source ML models on EC2.

  • SwinIR

    SwinIR: Image Restoration Using Swin Transformer (official repository)

    If you’re anything like us, though, you’ll feel compelled to poke around the code and run the model yourself. We’ll do that in this article using Meadowrun, an open-source library that makes it easy to run Python code in the cloud. For ML models in particular, we just added a feature for requesting GPU machines in a recent release. We’ll also feed the images generated by DALL·E Mini into additional image processing models (GLID-3-xl and SwinIR) to improve the quality of our generated images. Along the way we’ll deal with the speedbumps that come up when running open-source ML models on EC2.

  • dalle-playground

    A playground to generate images from any text prompt using Stable Diffusion (past: using DALL-E Mini)

    Next, we want the code in the https://github.com/hrichardlee/dalle-playground repo, and we want to construct a pip environment from the backend/requirements.txt file in that repo. We were almost able to use the saharmor/dalle-playground repo as-is, but we had to make one change to add the jax[cuda] package to the requirements.txt file. In case you haven’t seen jax before, jax is a machine-learning library from Google, roughly equivalent to Tensorflow or PyTorch. It combines Autograd for automatic differentiation and XLA (accelerated linear algebra) for JIT-compiling numpy-like code for Google’s TPUs or Nvidia’s CUDA API for GPUs. The CUDA support requires explicitly selecting the [cuda] option when we install the package.

  • dalle-playground

    A playground to generate images from any text prompt using DALL-E Mini and based on OpenAI's DALL-E https://openai.com/blog/dall-e/ (by hrichardlee)

    Next, we want the code in the https://github.com/hrichardlee/dalle-playground repo, and we want to construct a pip environment from the backend/requirements.txt file in that repo. We were almost able to use the saharmor/dalle-playground repo as-is, but we had to make one change to add the jax[cuda] package to the requirements.txt file. In case you haven’t seen jax before, jax is a machine-learning library from Google, roughly equivalent to Tensorflow or PyTorch. It combines Autograd for automatic differentiation and XLA (accelerated linear algebra) for JIT-compiling numpy-like code for Google’s TPUs or Nvidia’s CUDA API for GPUs. The CUDA support requires explicitly selecting the [cuda] option when we install the package.

  • jax

    Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

    Next, we want the code in the https://github.com/hrichardlee/dalle-playground repo, and we want to construct a pip environment from the backend/requirements.txt file in that repo. We were almost able to use the saharmor/dalle-playground repo as-is, but we had to make one change to add the jax[cuda] package to the requirements.txt file. In case you haven’t seen jax before, jax is a machine-learning library from Google, roughly equivalent to Tensorflow or PyTorch. It combines Autograd for automatic differentiation and XLA (accelerated linear algebra) for JIT-compiling numpy-like code for Google’s TPUs or Nvidia’s CUDA API for GPUs. The CUDA support requires explicitly selecting the [cuda] option when we install the package.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • autograd

    Efficiently computes derivatives of numpy code.

    Next, we want the code in the https://github.com/hrichardlee/dalle-playground repo, and we want to construct a pip environment from the backend/requirements.txt file in that repo. We were almost able to use the saharmor/dalle-playground repo as-is, but we had to make one change to add the jax[cuda] package to the requirements.txt file. In case you haven’t seen jax before, jax is a machine-learning library from Google, roughly equivalent to Tensorflow or PyTorch. It combines Autograd for automatic differentiation and XLA (accelerated linear algebra) for JIT-compiling numpy-like code for Google’s TPUs or Nvidia’s CUDA API for GPUs. The CUDA support requires explicitly selecting the [cuda] option when we install the package.

  • meadowrun-dallemini-demo

    A demo of using Meadowrun to run DALL·E Mini, GLID3-XL, and SwinIR in an image generation pipeline

    For the second half of this article, we’ll use meadowdata/meadowrun-dallemini-demo which contains a notebook for running multiple models as sequential batch jobs to generate images using Meadowrun. The combination of models is inspired by jina-ai/dalle-flow.

  • dalle-flow

    🌊 A Human-in-the-Loop workflow for creating HD images from text

    For the second half of this article, we’ll use meadowdata/meadowrun-dallemini-demo which contains a notebook for running multiple models as sequential batch jobs to generate images using Meadowrun. The combination of models is inspired by jina-ai/dalle-flow.

  • meadowrun_dallemini_demo

    Discontinued A demo of using Meadowrun to run DALL·E Mini, GLID3-XL, and SwinIR in an image generation pipeline [Moved to: https://github.com/meadowdata/meadowrun-dallemini-demo]

    We’ll then need to open the main notebook in Jupyter, and edit S3_BUCKET_NAME and S3_BUCKET_REGION to match the bucket we created in the first half of this article.

  • warehouse

    The Python Package Index

    We’re referencing our three models directly as git repos (because they’re not available as packages in PyPI) in model_requirements.txt, but we had to make a few changes to make these repos work as pip packages. Pip looks for a setup.py file in the git repo to figure out which files from the repo need to be installed into the environment, as well as what the dependencies of that repo are. GLID-3-xl and latent-diffusion (another diffusion model that GLID-3-xl depends on) had setup.py files that needed tweaks to include all of the code needed to run the models. SwinIR didn’t have a setup.py file at all, so we added one. Finally, all of these setup.py files needed additional dependencies, which we just added to the model_requirements.txt file.

  • latent-diffusion

    High-Resolution Image Synthesis with Latent Diffusion Models

    We’re referencing our three models directly as git repos (because they’re not available as packages in PyPI) in model_requirements.txt, but we had to make a few changes to make these repos work as pip packages. Pip looks for a setup.py file in the git repo to figure out which files from the repo need to be installed into the environment, as well as what the dependencies of that repo are. GLID-3-xl and latent-diffusion (another diffusion model that GLID-3-xl depends on) had setup.py files that needed tweaks to include all of the code needed to run the models. SwinIR didn’t have a setup.py file at all, so we added one. Finally, all of these setup.py files needed additional dependencies, which we just added to the model_requirements.txt file.

  • glid-3-xl

    1.4B latent diffusion model fine tuning (by hrichardlee)

    We’re referencing our three models directly as git repos (because they’re not available as packages in PyPI) in model_requirements.txt, but we had to make a few changes to make these repos work as pip packages. Pip looks for a setup.py file in the git repo to figure out which files from the repo need to be installed into the environment, as well as what the dependencies of that repo are. GLID-3-xl and latent-diffusion (another diffusion model that GLID-3-xl depends on) had setup.py files that needed tweaks to include all of the code needed to run the models. SwinIR didn’t have a setup.py file at all, so we added one. Finally, all of these setup.py files needed additional dependencies, which we just added to the model_requirements.txt file.

  • latent-diffusion

    High-Resolution Image Synthesis with Latent Diffusion Models (by hrichardlee)

    We’re referencing our three models directly as git repos (because they’re not available as packages in PyPI) in model_requirements.txt, but we had to make a few changes to make these repos work as pip packages. Pip looks for a setup.py file in the git repo to figure out which files from the repo need to be installed into the environment, as well as what the dependencies of that repo are. GLID-3-xl and latent-diffusion (another diffusion model that GLID-3-xl depends on) had setup.py files that needed tweaks to include all of the code needed to run the models. SwinIR didn’t have a setup.py file at all, so we added one. Finally, all of these setup.py files needed additional dependencies, which we just added to the model_requirements.txt file.

  • SwinIR

    SwinIR: Image Restoration Using Swin Transformer (official repository) (by hrichardlee)

    We’re referencing our three models directly as git repos (because they’re not available as packages in PyPI) in model_requirements.txt, but we had to make a few changes to make these repos work as pip packages. Pip looks for a setup.py file in the git repo to figure out which files from the repo need to be installed into the environment, as well as what the dependencies of that repo are. GLID-3-xl and latent-diffusion (another diffusion model that GLID-3-xl depends on) had setup.py files that needed tweaks to include all of the code needed to run the models. SwinIR didn’t have a setup.py file at all, so we added one. Finally, all of these setup.py files needed additional dependencies, which we just added to the model_requirements.txt file.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts