16 papers
MOSAIC: Adaptive Inter-layer Composition for Efficient Heterogeneous Vision-Language Models
Yuncheng Yang, Feiyang Ye, Shixian Luo +7
Vision-Language Models (VLMs) have achieved success using homogeneous Transformers to process multimedia data. Recent studies show that heterogeneous structures interleaving effici…
C3-UniMM: Causal Cycle-Consistent Unified Multimodal Modeling via Super Alignment and Shared Decoding Space
Yujie Shen, Lianlei Shan
Unified Multimodal Models aim to achieve any-to-any understanding and generation across arbitrary modalities. However, existing methods primarily rely on modeling implicit statisti…
PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space
Bochen Yang, Lianlei Shan
Current Vision-Language-Action (VLA) models face a trade-off between efficient action generation and explicit deliberation. Directly decoding actions from vision-language backbone…
DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
David Huang, Lianlei Shan
Reasoning capabilities of multimodal large language models (MLLMs) have improved considerably in recent years. Existing approaches typically rely on explicit chain-of-thought or co…
Think Less, Act Early: Reinforced Latent Reasoning with Early Exit in Vision-Language-Action Models
Dianqiao Lei, Lianlei Shan
Existing Vision-Language-Action (VLA) models predominantly rely on explicit Chain-of-Thought (CoT) reasoning to bridge perception and action. While effective, this paradigm suffers…
-Prediction Flow: Efficient Continuous Decoding for Masked Diffusion Language Models
Weitian Wang, Lianlei Shan, Shubham Rai +2
Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens, but their standard decoder reduces each step to a binary action: a position is either commit…
MPCoT: Reward-Guided Multi-Path Latent Reasoning for Test-Time Scalable Vision-Language-Action
Boyang Zhang, Lianlei Shan
Vision-Language-Action (VLA) policies remain brittle in long-horizon control, where one-pass action decoding offers limited inference-time deliberation. Explicit chain-of-thought a…
MSD-Score: Multi-Scale Distributional Scoring for Reference-Free Image Caption Evaluation
Shichao Kan, Xuyang Zhang, Haojie Zhang +7
Evaluating image captions without references remains challenging because global embedding similarity often misses fine-grained mismatches such as hallucinated objects, missing attr…
Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization
Wengao Ye, Yan Liang, Lianlei Shan
Recent advancements in Large Language Models (LLMs) have shifted from explicit Chain-of-Thought (CoT) reasoning to more efficient latent reasoning, where intermediate thoughts are…
RecLLM-R1: A Two-Stage Training Paradigm with Reinforcement Learning and Chain-of-Thought v1
Yu Xie, Xingkai Ren, Ying Qi +2
Traditional recommendation systems often grapple with "filter bubbles", underutilization of external knowledge, and a disconnect between model optimization and business policy iter…
GeoLocSFT: Efficient Visual Geolocation via Supervised Fine-Tuning of Multimodal Foundation Models
Qiang Yi, Lianlei Shan
Accurately determining the geographic location where a single image was taken, visual geolocation, remains a formidable challenge due to the planet's vastness and the deceptive sim…
GMM-Based Comprehensive Feature Extraction and Relative Distance Preservation For Few-Shot Cross-Modal Retrieval
Chengsong Sun, Weiping Li, Xiang Li +2
Few-shot cross-modal retrieval focuses on learning cross-modal representations with limited training samples, enabling the model to handle unseen classes during inference. Unlike t…
GeoGramBench: Benchmarking the Geometric Program Reasoning in Modern LLMs
Shixian Luo, Zezhou Zhu, Yu Yuan +3
Geometric spatial reasoning forms the foundation of many applications in artificial intelligence, yet the ability of large language models (LLMs) to operate over geometric spatial…
LLM-CoT Enhanced Graph Neural Recommendation with Harmonized Group Policy Optimization
Hailong Luo, Bin Wu, Hongyong Jia +2
Graph neural networks (GNNs) have advanced recommender systems by modeling interaction relationships. However, existing graph-based recommenders rely on sparse ID features and do n…
Cognitive Memory in Large Language Models
Lianlei Shan, Shixian Luo, Zezhou Zhu +2
This paper examines memory mechanisms in Large Language Models (LLMs), emphasizing their importance for context-rich responses, reduced hallucinations, and improved efficiency. It…
DynRsl-VLM: Enhancing Autonomous Driving Perception with Dynamic Resolution Vision-Language Models
Xirui Zhou, Lianlei Shan, Xiaolin Gui
Visual Question Answering (VQA) models, which fall under the category of vision-language models, conventionally execute multiple downsampling processes on image inputs to strike a…