10 papers
UniCon-Former: Unified Convolution Transformer is All You Need for Hand Gesture Recognition
Mallika Garg, Debashis Ghosh, Pyari Mohan Pradhan
Convolutional Neural Networks (CNNs) capture local features efficiently but struggle with global context due to their limited receptive field. On the other hand, transformers effec…
Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models
Ruihan Xu, Yuting Gao, Lan Wang +5
Large Multimodal Models (LMMs) have achieved remarkable success in vision-language tasks, yet their vast parameter counts are often underutilized during both training and inference…
FlattenGPT: Depth Compression for Transformer with Layer Flattening
Ruihan Xu, Qingpei Guo, Yao Zhu +3
Recent works have indicated redundancy across transformer blocks, prompting the research of depth compression to prune less crucial blocks. However, current ways of entire-block pr…
Social Debiasing for Fair Multi-modal LLMs
Harry Cheng, Yangyang Guo, Qingpei Guo +4
Multi-modal Large Language Models (MLLMs) have dramatically advanced the research field and delivered powerful vision-language understanding capabilities. However, these models oft…
From Mapping to Composing: A Two-Stage Framework for Zero-shot Composed Image Retrieval
Yabing Wang, Zhuotao Tian, Qingpei Guo +4
Composed Image Retrieval (CIR) is a challenging multimodal task that retrieves a target image based on a reference image and accompanying modification text. Due to the high cost of…
M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance
Qingpei Guo, Kaiyou Song, Zipeng Feng +9
We present M2-omni, a cutting-edge, open-source omni-MLLM that achieves competitive performance to GPT-4o. M2-omni employs a unified multimodal sequence modeling framework, which e…