8 papers · 1 filter
Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformers
Xinyu Peng, Han Li, Yuyang Huang +7
Existing video frame interpolation (VFI) methods often adopt a frame-centric approach, processing videos as independent short segments (e.g., triplets), which leads to temporal inc…
OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
Han Li, Xinyu Peng, Yaoming Wang +7
We introduce OneCAT, a unified multimodal model that seamlessly integrates understanding, generation, and editing within a novel, pure decoder-only transformer architecture. Our fr…
METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Models
Yuchen Liu, Yaoming Wang, Bowen Shi +5
Vision encoders serve as the cornerstone of multimodal understanding. Single-encoder architectures like CLIP exhibit inherent constraints in generalizing across diverse multimodal…
Noise Conditional Variational Score Distillation
Xinyu Peng, Ziyang Zheng, Yaoming Wang +6
We propose Noise Conditional Variational Score Distillation (NCVSD), a novel method for distilling pretrained diffusion models into generative denoisers. We achieve this by reveali…
Learning Unsupervised Gaze Representation via Eye Mask Driven Information Bottleneck
Yangzhou Jiang, Yinxin Lin, Yaoming Wang +3
Appearance-based supervised methods with full-face image input have made tremendous advances in recent gaze estimation tasks. However, intensive human annotation requirement inhibi…
Parameter-efficient Fine-tuning in Hyperspherical Space for Open-vocabulary Semantic Segmentation
Zelin Peng, Zhengqin Xu, Zhilin Zeng +2
Open-vocabulary semantic segmentation seeks to label each pixel in an image with arbitrary text descriptions. Vision-language foundation models, especially CLIP, have recently emer…