4 papers
Accelerating Vision Transformers with Adaptive Patch Sizes
Rohan Choudhury, JungEun Kim, Jinhyung Park +3
Vision Transformers (ViTs) partition input images into uniformly sized patches regardless of their content, resulting in long input sequence lengths for high-resolution images. We…
MAVERIX: Multimodal Audio-Visual Evaluation and Recognition IndeX
Liuyue Xie, Avik Kuthiala, George Z. Wei +12
We introduce MAVERIX (Multimodal audiovisual Evaluation and Recognition IndeX), a unified benchmark to probe the video understanding in multimodal LLMs, encompassing video, audio,…
SkipSR: Faster Super Resolution with Token Skipping
Rohan Choudhury, Shanchuan Lin, Jianyi Wang +6
Diffusion-based super-resolution (SR) is a key component in video generation and video restoration, but is slow and expensive, limiting scalability to higher resolutions and longer…
Don't Look Twice: Faster Video Transformers with Run-Length Tokenization
Rohan Choudhury, Guanglei Zhu, Sihan Liu +3
Transformers are slow to train on videos due to extremely large numbers of input tokens, even though many video tokens are repeated over time. Existing methods to remove such uninf…