Xiaomi Introduces PROVE Metrics for Evaluating Object Removal
A team from MiLM Plus at Xiaomi Inc. has released PROVE, a framework for evaluating object-removal models. Accepted at ACM MM 2026, it addresses the difficulty of judging an ill-posed task in which multiple restorations can plausibly fill the same missing region. PROVE uses two reference-free metrics: RC-S measures spatial coherence, while RC-T evaluates temporal consistency in video. Both analyze local distributions of DINOv2 features with sliding-window Maximum Mean Discrepancy rather than comparing results with a single ground-truth image or video.
The accompanying PROVE-Bench contains PROVE-M, with 80 paired, motion-augmented videos, and PROVE-H, with 100 challenging videos without ground truth. Against rankings from 20 participants, RC-S achieved average Kendall’s τ of 0.59, compared with 0.26 for ReMOVE and 0.16 for CFD, and ranked first on five of six benchmarks. On RORD-Val, it preferred clean images over blurred or region-swapped versions in 100% of tests.
The Apache 2.0 PyTorch implementation requires Python 3.10+, PyTorch 2.6+, Transformers 4.51+ and DINOv2-giant weights. RC-S runs at 134.6 ms per frame on an RTX 4090. The authors note that the method is intended for evaluation and is not suited to real-time on-device scoring or effects extending beyond the evaluation crop.
Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.