eg3d VS PanoHead

It's based on this, which has cats https://github.com/NVlabs/eg3d

EG3D is not a new idea, but developers claim that it works significantly faster and more effective than other neural networks. It is open source, you can test it on GitHub (https://github.com/NVlabs/eg3d).

Continue reading | Checkout the paper, github

So we can finally play around with the cool NVLabs EG3D, but they refuse to release the inversion script.

Using the recently released PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360° code https://github.com/sizhean/panohead I had a go with people generated from Stable Diffusion (using DeliberateV2 model). I've not figured out yet how they're generating the camera pose information (in the dataset.json inside dataset folder) so I just used their example images and ran through ControlNet to ensure it was the same camera pose for my images, swapping my image file name for the label in the json array. To get the code running I used WSL Ubuntu and installed Cuda Toolkit 11.3, then created the conda environment from their instructions. I think the code is expecting multiple GPU so I had to change the line device = torch.device('cuda') in all of the main python files in the repo. It could be possible to use the results of this to make a synthetic dataset for dreambooth training of a new coherent person (with a bit of work, the quality needs to be a little higher, maybe running images back through img2img + ControlNet first to clean up).