pix2pix VS stylegan

I have previously worked on retraining Pix2Pix GAN for image-to-image style transfer retrained with image-mask pairs. I expect Stable Diffusion to be better than Pix2Pix, but the problem sounds like something that should have been tackled already. I am familiar with text-based instructions for style transfer using SD (Instruct Pix2Pix), but retraining with image-mask pairs should provide better results. Does anyone know if anything that like exists already? Reference for Pix2Pix https://phillipi.github.io/pix2pix/

Additionally, there are some open source AI models for texture generation, such as pix2pix: https://phillipi.github.io/pix2pix/.

Sounds fun. Also sounds like image-to-image translation in 1D? Pix2pix is a famous implementation that uses UNet + adversarial loss: https://github.com/phillipi/pix2pix

I guess everything is rasterized correctly. While u/Kelaifu is correct: there are no softwares that can guess your face where it's missing, right now there are machine learning techniques and implementations (see: https://phillipi.github.io/pix2pix/) that can go pretty far with guesses.

[Image generated using Machine Learning algorithm pix2pix trained by me using this dataset: CMP Facade Dataset]

pix2pix can already transform images between satellite and maps (https://phillipi.github.io/pix2pix/). Will there be any difference if I use pix2pix to transform satellite images to maps and back?

# Resize all images in a folder by 1/2 # modified from https://github.com/phillipi/pix2pix/blob/master/scripts/combine_A_and_B.py # resize_A_to_C.py from pdb import set_trace as st import os import numpy as np import cv2 import argparse parser = argparse.ArgumentParser("resize images by 1/2") parser.add_argument( "--fold_A", dest="fold_A", help="input directory for original images", type=str, default="./test_A", ) parser.add_argument( "--fold_C", dest="fold_C", help="output directory", type=str, default="./test_C" ) args = parser.parse_args() for arg in vars(args): print("[%s] = " % arg, getattr(args, arg)) img_list = os.listdir(args.fold_A) if not os.path.isdir(args.fold_C): os.makedirs(args.fold_C) for name_A in img_list: path_A = os.path.join(args.fold_A, name_A) if os.path.isfile(path_A) and not name_A.startswith( "." ): # skip . and .. and folders im_A = cv2.imread(path_A, cv2.IMREAD_COLOR) # Example: scale_down by factor 1/2 both dimensions scale_down = 0.5 im_C = cv2.resize( im_A, None, fx=scale_down, fy=scale_down, interpolation=cv2.INTER_LINEAR ) # store resized image with same name as in folder A path_C = os.path.join(args.fold_C, name_A) cv2.imwrite(path_C, im_C)

This project uses the pix2pix Image translation architecture (https://phillipi.github.io/pix2pix/) for 3D image post-processing. The goal was to test the 3D applications of this type of architecture.

It seems like this even would make for a cool website, like pix2pix

Been following generative AI since 2017 when nvidia released their first GAN paper & the results always fascinated me. Trained my own models with their repo then experimented with other open source projects. went thru the pain of assembling my own data set, tweaking code parameters to achieve what i'm looking for, had to deal with all kinds of hardware/software issues. I know it's not easy. (screenshot of a motorbike GAN model i was training in 2018 https://imgur.com/a/SIULFhR, was taken after 5 hours of training on a gtx 1080) or this, cinema camera output from another locally trained model. So yeah i have a couple ideas of how generative AI works. yup things were that bad few years ago, that technology has come a long way. Using & setting up something like stable diffusion with automatic1111 webui isn't really a complex process. Though generating AI art locally is always gonna feel more rewarding than using a cloud based service.

Their algorithm is public, you could do it yourself if you have the proper hardware: https://github.com/NVlabs/stylegan

Umm, StyleGAN was the first decent image generation model, and it was producing great images from random seeds 5 years ago. Now, that's with the obvious caveat that each model was trained to produce one specific type of image and it helped immensely if the training images were all aligned the same. Diffusion models are certainly the trendy current architecture for image generation, but AFAIK there's no fundamental theoretical limitation to the output quality of any architecture except the general rule that more parameters is better.

Jokes aside, ML moves extremely fast and our field is quickly advancing. The honest truth is that no researcher can even keep up other than their extremely niche corner. I'll show you an example. Here's what state of the art image generation looked like in 2014, 2018, and here is today (which now is highly controllable using text prompts instead of data prompts).

The fundamental technology we will use in this work is a generative adversarial network. Specifically, the Style GAN variant.

Based on my findings, we don't really need FP64 unless it's for certain medical applications. But The Best GPUs for Deep Learning in 2020 — An In-depth Analysis is suggesting A100 outperforms A6000 ~50% in DL. Also the Stylegan project  GitHub - NVlabs/stylegan: StyleGAN - Official TensorFlow Implementation uses NVIDIA DGX-1 with 8 Tesla V100 16G(Fp32=15TFLOPS) to train dataset of  high-res 1024*1024 images, I'm getting a bit uncertain if my specific tasks would require FP64 since my dataset is also high-res images. If not, can I assume A6000*5(total 120G) could provide similar results for StyleGan?

Yes that's what I thought. But StyleGan https://github.com/NVlabs/stylegan uses NVIDIA DGX-1 with 8 Tesla V100 16G GPUs(FP32=15) to do the training, not sure if it's related to its high-res training images or something else.