4 papers
Channel-wise Vector Quantization
Wei Song, Tianhang Wang, Yitong Chen +5
We present Channel-wise Vector Quantization (CVQ), a novel image tokenization paradigm that replaces patch-wise tokens with channel-wise tokens. Unlike conventional vector quantiza…
Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini
Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang +86
We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage t…
NudgeVAD: Language-Nudged End-to-End Driving via FiLM Residuals
Chieh-Chi Yang, Yu-Hsiang Chen, Yi-Ting Chen
Natural-language instructions promise controllable end-to-end driving, but their benefit can be hidden when planners already receive reliable high-level commands. We propose NudgeV…
Unison: A Fully Automatic, Task-Universal, and Low-Cost Framework for Unified Understanding and Generation
Shihao Zhao, Yitong Chen, Zeyinzi Jiang +5
Unified understanding and generation is a highly appealing research direction in multimodal learning. There exist two approaches: one trains a transformer via an auto-regressive pa…