4 papers
Studying Image Tokenizers as Visual Languages in Unified Multimodal Models
Siting Li, Zhengyang Wang, Simon Shaolei Du +2
Image tokenizers define the ``visual language'' of unified multimodal models, yet are commonly studied through isolated metrics or generation-/understanding-only evaluations. These…
RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
Zhiyuan Zeng, Hamish Ivison, Yiping Wang +14
We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide alg…
Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image Retrieval
Siting Li, Xiang Gao, Simon Shaolei Du
While an image is worth more than a thousand words, only a few provide crucial information for a given task and thus should be focused on. In light of this, ideal text-to-image (T2…
Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder
Siting Li, Pang Wei Koh, Simon Shaolei Du
Recent research has shown that CLIP models struggle with visual reasoning tasks that require grounding compositionality, understanding spatial relationships, or capturing fine-grai…