helpful pointers to state-of-the-art material for depth estimation from multi-view videos captured from cameras with arbitrary poses.

• colmap

  28 6,720 9.2 C++

  COLMAP - Structure-from-Motion and Multi-View Stereo

  

I am looking for some pointers to papers/open-source software that can do fast and accurate depth estimation from multi-view videos. I am aware of colmap and OpenMVG software but was curious to know if there are others out there that outperform these. I am also interested in any state-of-the-art for depth estimation using unsupervised or self-supervised neural networks.

• OpenMVG (open Multiple View Geometry)

  10 5,457 6.6 C++

  open Multiple View Geometry library. Basis for 3D computer vision and Structure from Motion.

  

I am looking for some pointers to papers/open-source software that can do fast and accurate depth estimation from multi-view videos. I am aware of colmap and OpenMVG software but was curious to know if there are others out there that outperform these. I am also interested in any state-of-the-art for depth estimation using unsupervised or self-supervised neural networks.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• BundleFusion

  1 1,449 10.0 C++

  [Siggraph 2017] BundleFusion: Real-time Globally Consistent 3D Reconstruction using Online Surface Re-integration

Take a look at BundleFusion: Paper, Project Page, Source Code and SIFTGPU Source Code, used in BundleFusion source. It has a “server GPU” implementation

• SiftGPU

  1 355 10.0 C++

Take a look at BundleFusion: Paper, Project Page, Source Code and SIFTGPU Source Code, used in BundleFusion source. It has a “server GPU” implementation

• ACMMP

  1 162 0.0 C++

  Multi-Scale Geometric Consistency Guided and Planar Prior Assisted Multi-View Stereo (TPAMI 2022)

That said, for classical 3D reconstruction from images, COLMAP is by far the best method in terms of usability - performances tradeoff. You can get something a bit better (such as https://github.com/GhiXu/ACMMP), but it is not very user-friendly.

• deep-video-mvs

  1 198 0.0 Python

  Code for "DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion" (CVPR 2021)

If deep learning is an option, then you might want to check out http://zak.murez.com/atlas/, https://zju3dv.github.io/neuralrecon/, https://github.com/ardaduz/deep-video-mvs and the references therein. These methods can be better than classical ones, especially if overfitted on a specific type of scene.

• nerf-pytorch

  3 4,998 0.0 Python

  A PyTorch implementation of NeRF (Neural Radiance Fields) that reproduces the results.

I have been curious about NeRF and have tried out the software. I really like the idea, but I have found generating the NeRF representation to be too slow and does not do seem to do too well if the number of cameras in the rig are a bit sparse. I have seen that there have been some evolution and variations of the NeRF concept and I am looking into some of them currently.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• instant-ngp

  147 15,260 6.7 Cuda

  Instant neural graphics primitives: lightning fast NeRF and more

  

Regarding Nerf being too slow, you might check out instant nerf from nvidia which has nerf models training in seconds to minutes.

• depthai

  1 863 9.0 Python

  DepthAI Python API utilities, examples, and tutorials.

  

ah, good call-out. Luxonis' DepthAI has solid resources for this here: https://github.com/luxonis/depthai/issues/173 DepthAI runs neural inference on stereo to produce a depth map (among doing other things too)

Suggest a related project