11 papers
Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction
Zhongbin Guo, Jiahao Xie, Dongling Xiao +5
While Multimodal Large Language Models (MLLMs) have achieved remarkable progress, visual understanding and generation are typically treated as divergent objectives. Existing unifie…
PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis
Xiaomin He, Dongling Xiao, Jiahao Xie +4
Real-world multimodal instructions often bundle multiple requirements with unequal importance, yet most multimodal training data still reduce instruction following to answering one…
DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes
Jiahao Xie, Zhongbin Guo, Qianle Wang +4
While data curation for Vision Language Models (VLMs) is increasingly active, public practice for constructing pretraining mixtures remains largely heuristic: practitioners stack d…
How Do LLMs and VLMs Understand Viewpoint Rotation Without Vision? An Interpretability Study
Zhen Yang, Ping Jian, Zhongbin Guo +5
Over the past year, spatial intelligence has drawn increasing attention. Many prior works study it from the perspective of visual-spatial intelligence, where models have access to…
The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs
Yonghong Deng, Zhen Yang, Ping Jian +4
With the rapid advancement of large language models (LLMs), the safety of LLMs has become a critical concern. Despite significant efforts in safety alignment, current LLMs remain v…
Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
Jiacheng Wang, Ping Jian, Zhijie Liu +5
Large language models are commonly adapted to downstream tasks through supervised fine-tuning (SFT), but substantial distribution mismatch between downstream supervision and a mode…
Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions
Zhongbin Guo, Zhen Yang, Yushan Li +6
Recent advancements in Spatial Intelligence (SI) have predominantly relied on Vision-Language Models (VLMs), yet a critical question remains: does spatial understanding originate f…
LISA-3D: Lifting Language-Image Segmentation to 3D via Multi-View Consistency
Zhongbin Guo, Jiahe Liu, Wenyu Gao +4
Text-driven 3D reconstruction requires masks that understand free-form instructions and remain stable under viewpoint changes. We present LISA-3D, a two-stage framework that adapts…
Beyond Flatlands: Unlocking Spatial Intelligence by Decoupling 3D Reasoning from Numerical Regression
Zhongbin Guo, Jiahe Liu, Yushan Li +5
Existing Vision Language Models (VLMs) architecturally rooted in "flatland" perception, fundamentally struggle to comprehend real-world 3D spatial intelligence. This failure stems…
Understanding Temporal Logic Consistency in Video-Language Models through Cross-Modal Attention Discriminability
Chengzhi Li, Heyan Huang, Ping Jian +3
Large language models (LLMs) often generate self-contradictory outputs, which severely impacts their reliability and hinders their adoption in practical applications. In video-lang…
TAMMs: Change Understanding and Forecasting in Satellite Image Time Series with Temporal-Aware Multimodal Models
Zhongbin Guo, Yuhao Wang, Ping Jian +4
Temporal Change Description (TCD) and Future Satellite Image Forecasting (FSIF) are critical, yet historically disjointed tasks in Satellite Image Time Series (SITS) analysis. Both…