6 papers · 1 filter
Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Models
Tianfan Peng, Yuntao Du, Pengzhou Ji +9
Large multimodal models (LMMs) often suffer from severe inference inefficiency due to the large number of visual tokens introduced by image encoders. While recent token compression…
MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models
Kailin Jiang, Ning Jiang, Yuntao Du +8
Large Multimodal Models (LMMs) encode rich factual knowledge via cross-modal pre-training, yet their static representations struggle to maintain an accurate understanding of time-s…
KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls
Kailin Jiang, Hongbo Jiang, Ning Jiang +7
Large Multimodal Models encode extensive factual knowledge in their pre-trained weights. However, its knowledge remains static and limited, unable to keep pace with real-world deve…
Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models
Yifan Jia, Kailin Jiang, Yuyang Liang +11
Large Multimodal Models(LMMs) face notable challenges when encountering multimodal knowledge conflicts, particularly under retrieval-augmented generation(RAG) frameworks where the…
When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations
Kailin Jiang, Yuntao Du, Yukai Ding +7
Large Multimodal Models (LMMs) store vast amounts of pretrained knowledge but struggle to remain aligned with real-world updates, making it difficult to avoid capability degradatio…
MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge
Yuntao Du, Kailin Jiang, Zhi Gao +4
Knowledge editing techniques have emerged as essential tools for updating the factual knowledge of large language models (LLMs) and multimodal models (LMMs), allowing them to corre…