SuperGluePretrainedNetwork VS aquamam

Code: https://github.com/magicleap/SuperGluePretrainedNetwork

This basically includes a deep learning based approach to do keypoint detection, and match them across image frames. This includes papers like SuperPoint, Superglue, and more. There is also a way to do dense matching with neural networks.

Feature matching is the way to go imo. Try out OpenCV's inbuilt feature matching methods like SIFT and FLANN. If the performance is poor, you can even try out CNN aided matching algos like SuperGlue Link (CVPR2020)

SIFT and brute force matching is your best bet in classical computer vision if you're unconcerned with runtime. There are methods from deep learning that can perform better, somewhat domain dependent. Check out superpoint and superglue from magic leap. https://github.com/magicleap/SuperGluePretrainedNetwork

This paper is near and dear to my heart because I saw the presentation at the first and only machine learning conference I've ever attended (thanks a lot, ICML, NeurIPS (x2), and ICLR!). It's a neural rendering approach that precedes NeRF, but you can see some similarities (even more so in the follow-up paper about "Scene Representation Networks"). Sitzmann and co-authors also published a paper about using sinusoidal activations in implicit representation models at NeurIPS 2020, the same conference where the "Fourier Features" paper (which has many of the same authors as the NeRF paper) was also presented. It's always interesting to me to see how ideas in science often pop-up at the same time from different researchers (e.g., attention).

Code for the paper can be found here.