Homography-Based Loss Function for Camera Pose Regression

Homography-Based Loss Function for Camera Pose Regression ℒ 🟰 | 📸➖📷| ²

Joint work with Clémentin Boittiaux, Claire Dune, Aurélien Arnaubec and Vincent Hugel.

Paper: https://arxiv.org/abs/2205.01937

#ComputerVision #AI #DL #VisualLocalization #AR #SLAM #Robotics

In many computer vision tasks we need to embed into a scalar the difference between two 6DOF camera poses, whether we are refining view poses in #NeRF or performing visual localization in #AR or #robotics navigation.

Many solutions arbitrarily weigh the relative importance between position and orientation errors, require careful initialisation and additional heuristics to avoid instability, or present unwanted local minima.

We propose an approach based on the homographies between camera views (defined by the translation and rotation between cameras). When poses are equal, the homography becomes the identity. We define the proposed loss as the Frobenius norm of the difference between the homography and identity.

Furthermore we compute the integral of this error for all planes (parallel to one of the cameras) within two given distances.

The hyperparameters become physically interpretable or estimated from training data. They represent the distance range of scene observations to take into account in the error computation.

We show that the approach performs well in a DL-based visual localisation task on Cambridge Landmarks and 7-scenes datasets.

A PyTorch implementation is provided here: https://github.com/clementinboittiaux/homography-loss-function Only a few lines of code that can be easily integrated into existing pipelines.