3d-photo-inpainting VS BoostingMonocularDepth

If you are looking to create actual 3d models, the DepthMap extension does have a function to create PLY models with vertex color information, and to render clips with simple camera moves from that extracted 3d scene, including inpainting (as per the 3d-photo-inpainting paper)

The depthmap extension for A1111 has implemented the 3d-photo-inpainting code that is doing that kind of thing. That's what I used to use, first on a Colab, and then adapted for windows so I could run it locally. But it's much more convenient to do it directly from the Automatic1111 WebUI.

But things are even older than stable diffusion.

It's a slightly modified version of https://shihmengli.github.io/3D-Photo-Inpainting/

Thanks! I barely did anything though, just took a deep dream'ed photo made by another artist (Daniel Ambrosi) and passed it through this: https://shihmengli.github.io/3D-Photo-Inpainting/ (github and colab at bottom). Didn't even have to come up with the camera trajectory, was one of the presets in the repo

pretty sure it's this: https://github.com/vt-vl-lab/3d-photo-inpainting

I suggest using this instead: https://github.com/compphoto/BoostingMonocularDepth

Alternatively maybe run the original image through Boosting Monocular Depth / MiDAS to generate the height map and use that in Substance Designer to generate the other maps (I've only tried this with environment images, not textures) https://github.com/compphoto/BoostingMonocularDepth

With Boosting Monocular Depth with MidasAI or LeresAI you can batch process multiple images (frames) by default with the Colab. You just load them into the "input" folder. https://github.com/compphoto/BoostingMonocularDepth/blob/main/Boostmonoculardepth.ipynb (with the free version of Colab you might only be allowed a couple of hours a day). To download a bunch of depth map frames quickly you need to link your Google Drive to the Colab and then you can drag them directly into a Drive folder from the Colab. Also now there is a (Midas-based I think) After Effects solution. https://aescripts.com/depth-scanner/ but I get an error trying to run it with my hardware

Watch out for the depth model though. By default that uses BoostingMonocularDepth, which is adobe.

All the links over here https://github.com/compphoto/BoostingMonocularDepth