GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization

摘要

Visual localization methods often present a trade-off between the high efficiency of specialized approaches, such as scene coordinate regression, and the need for rich, versatile scene representations for broader robotics tasks. To bridge this gap, we explore the use of 3D Gaussian Splatting (3DGS), which enables a unified, photorealistic encoding of 3D geometry and appearance. We propose GSplatLoc, a self-contained framework that tightly integrates structure-based keypoint matching with rendering-based pose refinement. Our two-stage procedure first distills robust descriptors from the lightweight XFeat extractor into the 3DGS model, enabling coarse pose estimation via 2D-3D correspondences without external dependencies. In the second stage, the initial pose is refined by minimizing a photometric warp loss, which leverages the fast, differentiable rendering of 3DGS. Benchmarking on widely used indoor and outdoor datasets demonstrates state-of-the-art performance among neural rendering-based localization methods and highlights the framework’s robustness in challenging dynamic scenes. Project page: https://gsplatloc.github.io