42 citations · 125 across the 23 of their papers we have counts for
23 papers
Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models
Ruihan Xu, Yuting Gao, Lan Wang +5
Large Multimodal Models (LMMs) have achieved remarkable success in vision-language tasks, yet their vast parameter counts are often underutilized during both training and inference…
OrdMoE: Preference Alignment via Hierarchical Expert Group Ranking in Multimodal Mixture-of-Experts LLMs
Yuting Gao, Weihao Chen, Lan Wang +2
Preference learning has recently emerged as a pivotal strategy for post-training alignment of Multimodal Large Language Models (MLLMs). However, existing approaches predominantly r…
AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
Yuting Gao, Wang Lan, Hengyuan Zhao +3
Multimodal Mixture-of-Experts (MoE) models offer a promising path toward scalable and efficient large vision-language systems. However, existing approaches rely on rigid routing st…
Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation
Inclusion AI, :, Bowen Ma +73
We propose Ming-Flash-Omni, an upgraded version of Ming-Omni, built upon a sparser Mixture-of-Experts (MoE) variant of Ling-Flash-2.0 with 100 billion total parameters, of which on…
Ming-Omni: A Unified Multimodal Model for Perception and Generation
Inclusion AI, Biao Gong, Cheng Zou +55
We propose Ming-Omni, a unified multimodal model capable of processing images, text, audio, and video, while demonstrating strong proficiency in both speech and image generation. M…
EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Models
Linglin Jing, Yuting Gao, Zhigang Wang +5
Recent advancements have shown that the Mixture of Experts (MoE) approach significantly enhances the capacity of large language models (LLMs) and improves performance on downstream…
Panther: Illuminate the Sight of Multimodal LLMs with Instruction-Guided Visual Prompts
Honglin Li, Yuting Gao, Chenglu Zhu +3
Multimodal large language models (MLLMs) are closing the gap to human visual perception capability rapidly, while, still lag behind on attending to subtle images details or locatin…
Multi-Modal Prompt Learning on Blind Image Quality Assessment
Wensheng Pan, Timin Gao, Yan Zhang +10
Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Currently, leveraging seman…
RESTORE: Towards Feature Shift for Vision-Language Prompt Learning
Yuncheng Yang, Chuyan Zhang, Zuopeng Yang +6
Prompt learning is effective for fine-tuning foundation models to improve their generalization across a variety of downstream tasks. However, the prompts that are independently opt…
Sinkhorn Distance Minimization for Knowledge Distillation
Xiao Cui, Yulei Qin, Yuting Gao +7
Knowledge distillation (KD) has been widely adopted to compress large language models (LLMs). Existing KD methods investigate various divergence measures including the Kullback-Lei…
MMICT: Boosting Multi-Modal Fine-Tuning with In-Context Examples
Tao Chen, Enwei Zhang, Yuting Gao +5
Although In-Context Learning (ICL) brings remarkable performance gains to Large Language Models (LLMs), the improvements remain lower than fine-tuning on downstream tasks. This pap…
Towards Robust Text Retrieval with Progressive Learning
Tong Wu, Yulei Qin, Enwei Zhang +4
Retrieval augmentation has become an effective solution to empower large language models (LLMs) with external and verified knowledge sources from the database, which overcomes the…
SoftCLIP: Softer Cross-modal Alignment Makes CLIP Stronger
Yuting Gao, Jinfeng Liu, Zihan Xu +5
During the preceding biennium, vision-language pre-training has achieved noteworthy success on several downstream tasks. Nevertheless, acquiring high-quality image-text pairs, wher…
Efficient Decoder-free Object Detection with Transformers
Peixian Chen, Mengdan Zhang, Yunhang Shen +5
Vision transformers (ViTs) are changing the landscape of object detection approaches. A natural usage of ViTs in detection is to replace the CNN-based backbone with a transformer-b…
PyramidCLIP: Hierarchical Feature Alignment for Vision-language Model Pretraining
Yuting Gao, Jinfeng Liu, Zihan Xu +4
Large-scale vision-language pre-training has achieved promising results on downstream tasks. Existing methods highly rely on the assumption that the image-text pairs crawled from t…
DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning
Yuting Gao, Jia-Xin Zhuang, Shaohui Lin +4
While self-supervised representation learning (SSL) has received widespread attention from the community, recent research argue that its performance will suffer a cliff fall when t…
An Empirical Study and Analysis on Open-Set Semi-Supervised Learning
Huixiang Luo, Hao Cheng, Fanxu Meng +4
Pseudo-labeling (PL) and Data Augmentation-based Consistency Training (DACT) are two approaches widely used in Semi-Supervised Learning (SSL) methods. These methods exhibit great p…
Generative and Discriminative Learning for Distorted Image Restoration
Yi Gu, Yuting Gao, Jie Li +2
Liquify is a common technique for image editing, which can be used for image distortion. Due to the uncertainty in the distortion variation, restoring distorted images caused by li…
Association: Remind Your GAN not to Forget
Yi Gu, Jie Li, Yuting Gao +5
Neural networks are susceptible to catastrophic forgetting. They fail to preserve previously acquired knowledge when adapting to new tasks. Inspired by human associative memory sys…
Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning
Jinpeng Wang, Yuting Gao, Ke Li +7
Self-supervised learning has shown great potentials in improving the video representation ability of deep neural networks by getting supervision from the data itself. However, some…
Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion
Jinpeng Wang, Yuting Gao, Ke Li +5
One significant factor we expect the video representation learning to capture, especially in contrast with the image representation learning, is the object motion. However, we foun…
Double Supervised Network with Attention Mechanism for Scene Text Recognition
Yuting Gao, Zheng Huang, Yuchen Dai +3
In this paper, we propose Double Supervised Network with Attention Mechanism (DSAN), a novel end-to-end trainable framework for scene text recognition. It incorporates one text att…
Fused Text Segmentation Networks for Multi-oriented Scene Text Detection
Yuchen Dai, Zheng Huang, Yuting Gao +4
In this paper, we introduce a novel end-end framework for multi-oriented scene text detection from an instance-aware semantic segmentation perspective. We present Fused Text Segmen…