正在加载视频...
视频加载失败
Most multi-view reconstruction models need full supervision. We show they can self-improve without any ground truth labels. Introducing SelfEvo: Self-Improving 4D Perception via Self-Distillation. Up to +36.5% in video depth, +20.1% in camera estimation, zero annotation.
27,348 次观看 • 7 个月前 •via X (Twitter)
7 条评论

SelfEvo is driven by a very simple insight: the same model predicts better with richer context. This asymmetry creates a powerful self-distillation signal, letting the model improve itself on unlabeled data, without any additional supervision, almost like a free lunch.

Despite its simplicity, SelfEvo consistently improves pretrained multi-view reconstruction models: better on new domains, no drop on original ones, with gains that transfer to unseen domains, all without ground-truth annotations.

SelfEvo is flexible: it makes no static-scene assumption or multi-view consistency requirement, and significantly improves performance on dynamic scenes where annotations are most scarce. We visualize how model predictions on unseen dynamic scenes evolve during training.

Project page: Code: Led by @NanHuangg and @pengcheng_147, with @WeijiaZeng1, @RehgJim, @akanazawa, and @HavenFeng

Impressive gains with zero annotation. If this scales, it removes one of the biggest bottlenecks in multi view reconstruction: labeled data. Approaches like this could push 4D perception to learn directly from massive video streams.

This feels like a great idea and potentially applicable to many problems. Have people done something similar before?

Qian
