Track-Leakage-Free Hold-Out Self-Validation for Photogrammetric Reconstruction: Protocol, Sensitivity, and Limits

Abstract

Automated photogrammetric inspection emits metric measurements from a 3D reconstruction whose own correctness is normally unknown without an external survey. Can a reconstruction estimate its own reliability with no ground truth? We formalise a track-leakage-free hold-out protocol: a deterministic subset of images is withheld and each re-localised against only 3D points seen by at least two retained images -- a track-level barrier so a view is never tested against structure it helped create -- aggregated into an mAA confidence. Held-out observations still entered the bundle adjustment that built the trusted structure, so we call it track-leakage-free (a stricter re-mapping variant confirms this at 3/5 deg). Across operational GNSS-referenced captures plus ETH3D, EuRoC and the IMC 2025 benchmark we report four findings. (i) The protocol is computationally well-posed: good reconstructions score millidegree self-consistency. (ii) Thresholded self-consistency saturates and does not track absolute accuracy: confidence stays near 1.00 while true RTK error swings up to 14x within a capture, and the per-capture correlation is sign-unstable across five captures (-0.57 to +0.98; pooled 95% CI spanning zero), so it cannot gate accuracy. (iii) It flags gross failure only when the failure destroys internal consistency: a fragmenting model drops confidence, but a single self-consistent, globally-distorted model evades it -- three of four captures gave one model wrong by 55-106 m at confidence 1.00. (iv) The same dichotomy holds on independent ETH3D and IMC 2025 ground truth. Track-leakage-free hold-out measures internal geometric consistency, not absolute accuracy: neither a substitute for control-point assessment nor a general gross-failure gate. We release the protocol and degradation harness.