SlotMatch: Distilling Object-Centric Representations for Unsupervised Video Segmentation

Abstract

Unsupervised video segmentation is a challenging computer vision task, especially due to the lack of supervisory signals coupled with the complexity of visual scenes. To overcome this challenge, state-of-the-art models based on slot attention often have to rely on large and computationally expensive neural architectures. To this end, we propose a simple knowledge distillation framework that effectively transfers object-centric representations to a lightweight student. The proposed framework, called SlotMatch, aligns corresponding teacher and student slots via the cosine similarity, requiring no additional distillation objectives or auxiliary supervision. Unlike other distillation losses, our cosine-based distillation of slots enforces alignment of slot directions (which typically encode relational structure and semantic similarity), while ignoring their norms (which may encode confidence overestimation and dataset-specific scale artifacts). The simplicity of SlotMatch is supported by theoretical and empirical evidence, both indicating that integrating additional losses may be suboptimal. We conduct experiments on three datasets to compare state-of-the-art models, including our SlotContrast teacher, with our distilled student. The results show that our student based on SlotMatch matches and even outperforms its teacher, while using 3.6x fewer parameters and running up to 2.7x faster. Moreover, our student surpasses all other state-of-the-art unsupervised video segmentation models. We release our code at https://github.com/dianagrigore/SlotMatch.

Accepted at WACV 2027