DPT VS depthmap2mask

wget https://github.com/intel-isl/DPT/releases/download/1_0/dpt_large-midas-2f21e586.pt -O "C:\Users\itsju\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\Text To Image\Deforum Stable Diffusion\dpt_large-midas-2f21e586.pt" --tries=1 --no-check-certificate --progress=bar:force

try: from midas.dpt_depth import DPTDepthModel except: if not os.path.exists('MiDaS'): gitclone("https://github.com/isl-org/MiDaS.git") gitclone("https://github.com/bytedance/Next-ViT.git", f'{PROJECT_DIR}/externals/Next_ViT') if not os.path.exists('MiDaS/midas_utils.py'): shutil.move('MiDaS/utils.py', 'MiDaS/midas_utils.py') if not os.path.exists(f'{model_path}/dpt_large-midas-2f21e586.pt'): wget("https://github.com/intel-isl/DPT/releases/download/1_0/dpt_large-midas-2f21e586.pt", model_path) sys.path.append(f'{PROJECT_DIR}/MiDaS')

Try downloading the MiDaS model manually from here: https://github.com/intel-isl/DPT/releases/download/1_0/dpt_hybrid-midas-501f0c75.pt It should go in the stable-diffusion-webui/models/midas folder. If that doesn't work, try the stable-diffusion-webui/midas_models folder

Saving animation frames to output\2022-10\Test16 Downloading dpt_large-midas-2f21e586.pt... --------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) Cell In [16], line 550 548 # dispatch to appropriate renderer 549 if anim_args.animation_mode == '2D' or anim_args.animation_mode == '3D': --> 550 render_animation(args, anim_args) 551 elif anim_args.animation_mode == 'Video Input': 552 render_input_video(args, anim_args) Cell In [16], line 202, in render_animation(args, anim_args) 200 if predict_depths: 201 depth_model = DepthModel(device) --> 202 depth_model.load_midas(models_path) 203 if anim_args.midas_weight < 1.0: 204 depth_model.load_adabins() File ~\dsd\stable-diffusion\helpers\depth.py:38, in DepthModel.load_midas(self, models_path, half_precision) 36 if not os.path.exists(os.path.join(models_path, 'dpt_large-midas-2f21e586.pt')): 37 print("Downloading dpt_large-midas-2f21e586.pt...") ---> 38 wget("https://github.com/intel-isl/DPT/releases/download/1_0/dpt_large-midas-2f21e586.pt", models_path) 40 self.midas_model = DPTDepthModel( 41 path=f"{models_path}/dpt_large-midas-2f21e586.pt", 42 backbone="vitl16_384", 43 non_negative=True, 44 ) 45 normalization = NormalizeImage(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]) File ~\dsd\stable-diffusion\helpers\depth.py:16, in wget(url, outputdir) 15 def wget(url, outputdir): ---> 16 print(subprocess.run(['wget', url, '-P', outputdir], stdout=subprocess.PIPE).stdout.decode('utf-8')) File ~\miniconda3\envs\dsd\lib\subprocess.py:505, in run(input, capture_output, timeout, check, *popenargs, **kwargs) 502 kwargs['stdout'] = PIPE 503 kwargs['stderr'] = PIPE --> 505 with Popen(*popenargs, **kwargs) as process: 506 try: 507 stdout, stderr = process.communicate(input, timeout=timeout) File ~\miniconda3\envs\dsd\lib\subprocess.py:951, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask) 947 if self.text_mode: 948 self.stderr = io.TextIOWrapper(self.stderr, 949 encoding=encoding, errors=errors) --> 951 self._execute_child(args, executable, preexec_fn, close_fds, 952 pass_fds, cwd, env, 953 startupinfo, creationflags, shell, 954 p2cread, p2cwrite, 955 c2pread, c2pwrite, 956 errread, errwrite, 957 restore_signals, 958 gid, gids, uid, umask, 959 start_new_session) 960 except: 961 # Cleanup if the child failed starting. 962 for f in filter(None, (self.stdin, self.stdout, self.stderr)): File ~\miniconda3\envs\dsd\lib\subprocess.py:1420, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session) 1418 # Start the process 1419 try: -> 1420 hp, ht, pid, tid = _winapi.CreateProcess(executable, args, 1421 # no special security 1422 None, None, 1423 int(not close_fds), 1424 creationflags, 1425 env, 1426 cwd, 1427 startupinfo) 1428 finally: 1429 # Child is launched. Close the parent's copy of those pipe 1430 # handles that only the child should have open. You need (...) 1433 # pipe will not close when the child process exits and the 1434 # ReadFile will hang. 1435 self._close_pipe_fds(p2cread, p2cwrite, 1436 c2pread, c2pwrite, 1437 errread, errwrite) FileNotFoundError: [WinError 2] The system cannot find the file specified

wget("https://github.com/intel-isl/DPT/releases/download/1_0/dpt_large-midas-2f21e586.pt", models_path)

for change background i suggest to use depth2mask.

have you try with dept2mask?

Looks really good. Have you tried img2depth for the texturing? GitHub - Extraltodeus/depthmap2mask: Create masks out of depthmaps in img2img

have you try https://github.com/Extraltodeus/depthmap2mask ?

No, and I don't need to. depthmap2mask works sloppy, I don't like it. It's much better to create mask for "Inpainting" using image-editing software. Here you can see how it's done. https://www.youtube.com/watch?v=dnIYTGW1m8w

Sent to img2img and used Depth Aware img2img mask with the model set to `midas_v21_small` so that I would hopefully affect as little of the image as possible. (after seeing the pants morph, I think it might have been better to just use inpaint)