9 papers
Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation
Mohammadreza Sadeghi, Sareh Soleimani, Zihan Wang +1
Unsupervised Continual Learning (UCL) aims to enable neural networks to learn sequential tasks without labels or access to past data. A major challenge in this setting is Catastrop…
Closing the Indexing-Decoding Gap in Multimodal Generative Retrieval via Prefix Retention Optimization
Yufei Chen, Zihan Wang, Yubao Tang +3
Multimodal generative retrieval formulates multimodal retrieval as discrete identifier generation, eliminating the need for explicit similarity search over external embeddings. Exi…
MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG
Xihang Wang, Zihan Wang, Chengkai Huang +2
Multimodal Retrieval-Augmented Generation (MRAG) addresses key limitations of Multimodal Large Language Models (MLLMs), such as hallucination and outdated knowledge. However, curre…
Learning from Medical Entity Trees: An Entity-Centric Medical Data Engineering Framework for MLLMs
Jianghang Lin, Haihua Yang, Deli Yu +6
Multimodal Large Language Models (MLLMs) have shown transformative potential in medical applications, yet their performance is hindered by conventional data curation strategies tha…
MTRouter: Cost-Aware Multi-Turn LLM Routing with History-Model Joint Embeddings
Yiqun Zhang, Hao Li, Zihan Wang +6
Multi-turn, long-horizon tasks are increasingly common for large language models (LLMs), but solving them typically requires many sequential model invocations, accumulating substan…
ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding
Jovana Kondic, Pengyuan Li, Dhiraj Joshi +24
Understanding charts requires models to jointly reason over geometric visual patterns, structured numerical data, and natural language -- a capability where current vision-language…
LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoning
Haiying Xu, Zihan Wang, Song Dai +3
Despite recent advances in multimodal reasoning, representing auxiliary geometric constructions remains a fundamental challenge for multimodal large language models (MLLMs). Such c…
TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models
Harold Haodong Chen, Disen Lan, Wen-Jie Shu +10
The rapid evolution of video generative models has shifted their focus from producing visually plausible outputs to tackling tasks requiring physical plausibility and logical consi…
Video Finetuning Improves Reasoning Between Frames
Ruiqi Yang, Tian Yun, Zihan Wang +1
Multimodal large language models (LLMs) have made rapid progress in visual understanding, yet their extension from images to videos often reduces to a naive concatenation of frame…