Publications (63)
MASA: Rethinking the Representational Bottleneck in LoRA with Multi-A Shared Adaptation
Qin Dong, Yuntian Tang, Heming Jia +7
Low-Rank Adaptation (LoRA) has emerged as a dominant method in Parameter-Efficient Fine-Tuning (PEFT) for large language models, which augments the transformer layer with one down-…
Rethinking Centered Kernel Alignment in Knowledge Distillation
Zikai Zhou, Yunhang Shen, Shitong Shao +2
Knowledge distillation has emerged as a highly effective method for bridging the representation discrepancy between large-scale models and lightweight models. Prevalent approaches…
Weakly Supervised Open-Vocabulary Object Detection
Jianghang Lin, Yunhang Shen, Bingquan Wang +3
Despite weakly supervised object detection (WSOD) being a promising step toward evading strong instance-level annotations, its capability is confined to closed-set categories withi…
Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification
Xudong Tian, Zhizhong Zhang, Shaohui Lin +3
The Information Bottleneck (IB) provides an information theoretic principle for representation learning, by retaining all information relevant for predicting label while minimizing…
Aligning and Prompting Everything All at Once for Universal Visual Perception
Yunhang Shen, Chaoyou Fu, Peixian Chen +6
Vision foundation models have been explored recently to build general-purpose vision systems. However, predominant paradigms, driven by casting instance-level tasks as an object-wo…
Interleaving Reasoning for Better Text-to-Image Generation
Wenxuan Huang, Shuang Chen, Zheyong Xie +15
Unified multimodal understanding and generation models recently have achieve significant improvement in image generation capability, yet a large gap remains in instruction followin…
Novelty Detection via Contrastive Learning with Negative Data Augmentation
Chengwei Chen, Yuan Xie, Shaohui Lin +5
Novelty detection is the process of determining whether a query example differs from the learned training distribution. Previous methods attempt to learn the representation of the…
Data Upcycling Knowledge Distillation for Image Super-Resolution
Yun Zhang, Wei Li, Simiao Li +6
Knowledge distillation (KD) compresses deep neural networks by transferring task-related knowledge from cumbersome pre-trained teacher models to compact student models. However, cu…
CSD: Content-aware Speculative Decoding for Efficient Image Generation
Mingcheng Wang, Junbo Qiao, Yunchen Li +8
Speculative decoding (SD) has emerged as a key solution to accelerate the inference of autoregressive models. However, in the field of image generation, it faces the challenge of l…
IDMR: Towards Instance-Driven Precise Visual Correspondence in Multimodal Retrieval
Bangwei Liu, Yicheng Bao, Shaohui Lin +5
Multimodal retrieval systems are becoming increasingly vital for cutting-edge AI technologies, such as embodied AI and AI-driven digital content industries. However, current multim…
Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification
Wenxuan Huang, Zijie Zhai, Yunhang Shen +6
Multimodal Large Language Models (MLLMs) have achieved remarkable success in vision understanding, reasoning, and interaction. However, the inference computation and memory increas…
GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistant
Zhuokang Shen, Yifan Wang, Hanyu Chen +6
Recent advances in large language models (LLMs) have enabled increasingly capable chatbots. However, most existing systems focus on single-user settings and do not generalize well…
A General and Efficient Training for Transformer via Token Expansion
Wenxuan Huang, Yunhang Shen, Jiao Xie +5
The remarkable performance of Vision Transformers (ViTs) typically requires an extremely large training cost. Existing methods have attempted to accelerate the training of ViTs, ye…
A Closer Look at Branch Classifiers of Multi-exit Architectures
Shaohui Lin, Bo Ji, Rongrong Ji +1
Multi-exit architectures consist of a backbone and branch classifiers that offer shortened inference pathways to reduce the run-time of deep neural networks. In this paper, we anal…
Interpretable Neural Network Decoupling
Yuchao Li, Rongrong Ji, Shaohui Lin +5
The remarkable performance of convolutional neural networks (CNNs) is entangled with their huge number of uninterpretable parameters, which has become the bottleneck limiting the e…
MISSU: 3D Medical Image Segmentation via Self-distilling TransUNet
Nan Wang, Shaohui Lin, Xiaoxiao Li +4
U-Nets have achieved tremendous success in medical image segmentation. Nevertheless, it may suffer limitations in global (long-range) contextual interactions and edge-detail preser…
Hi-Mamba: Hierarchical Mamba for Efficient Image Super-Resolution
Junbo Qiao, Jincheng Liao, Wei Li +7
State Space Models (SSM), such as Mamba, have shown strong representation ability in modeling long-range dependency with linear complexity, achieving successful applications from h…
Towards Compact CNNs via Collaborative Compression
Yuchao Li, Shaohui Lin, Jianzhuang Liu +7
Channel pruning and tensor decomposition have received extensive attention in convolutional neural network compression. However, these two techniques are traditionally deployed in…
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
Bin Ren, Yawei Li, Nancy Mehta +129
This paper provides a comprehensive review of the NTIRE 2024 challenge, focusing on efficient single-image super-resolution (ESR) solutions and their outcomes. The task of this cha…
Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization
Peixian Chen, Kekai Sheng, Mengdan Zhang +5
Open-vocabulary object detection (OVD) aims to scale up vocabulary size to detect objects of novel categories beyond the training vocabulary. Recent work resorts to the rich knowle…
Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Models
Wenxuan Huang, Yu Zeng, Qiuchen Wang +14
Multimodal large language models (MLLMs) have achieved remarkable success across a broad range of vision tasks. However, constrained by the capacity of their internal world knowled…
SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space
Yunchen Li, Zhou Yu, Gaoqi He +4
Symmetric positive definite~(SPD) matrices have shown important value and applications in statistics and machine learning, such as FMRI analysis and traffic prediction. Previous wo…
Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler
Shaohui Lin, Wenxuan Huang, Jiao Xie +5
Filter pruning simultaneously accelerates the computation and reduces the memory overhead of CNNs, which can be effectively applied to edge devices and cloud services. In this pape…
A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise
Chaoyou Fu, Renrui Zhang, Zihan Wang +15
The surge of interest towards Multi-modal Large Language Models (MLLMs), e.g., GPT-4V(ision) from OpenAI, has marked a significant trend in both academia and industry. They endow L…
CompBench: Benchmarking Complex Instruction-guided Image Editing
Bohan Jia, Wenxuan Huang, Yuntian Tang +14
While real-world applications increasingly demand intricate scene manipulation, existing instruction-guided image editing benchmarks often oversimplify task complexity and lack com…
Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization
Mengtian Li, Shaohui Lin, Zihan Wang +3
Semi-supervised learning (SSL), thanks to the significant reduction of data annotation costs, has been an active research topic for large-scale 3D scene understanding. However, the…
Generation Properties of Stochastic Interpolation under Finite Training Set
Yunchen Li, Shaohui Lin, Zhou Yu
This paper investigates the theoretical behavior of generative models under finite training populations. Within the stochastic interpolation generative framework, we derive closed-…
LIPT: Latency-aware Image Processing Transformer
Junbo Qiao, Wei Li, Haizhen Xie +5
Transformer is leading a trend in the field of image processing. Despite the great success that existing lightweight image processing transformers have achieved, they are tailored…
The First Challenge on Mobile Real-World Image Super-Resolution at NTIRE 2026: Benchmark Results and Method Overview
Jiatong Li, Zheng Chen, Kai Liu +91
This paper provides a review of the NTIRE 2026 challenge on mobile real-world image super-resolution, highlighting the proposed solutions and the resulting outcomes. The challenge…
HUWSOD: Holistic Self-training for Unified Weakly Supervised Object Detection
Liujuan Cao, Jianghang Lin, Zebo Hong +4
Most WSOD methods rely on traditional object proposals to generate candidate regions and are confronted with unstable training, which easily gets stuck in a poor local optimum. In…
Complete Chess Games Enable LLM Become A Chess Master
Yinqi Zhang, Xintian Han, Haolong Li +2
Large language models (LLM) have shown remarkable abilities in text generation, question answering, language translation, reasoning and many other tasks. It continues to advance ra…
Training convolutional neural networks with cheap convolutions and online distillation
Jiao Xie, Shaohui Lin, Yichen Zhang +1
The large memory and computation consumption in convolutional neural networks (CNNs) has been one of the main barriers for deploying them on resource-limited systems. To this end,…
Towards Compact ConvNets via Structure-Sparsity Regularized Filter Pruning
Shaohui Lin, Rongrong Ji, Yuchao Li +2
The success of convolutional neural networks (CNNs) in computer vision applications has been accompanied by a significant increase of computation and memory costs, which prohibits…
A Theoretical Analysis of Memory and Overfitting Phenomena in Stochastic Interpolation Models
Yunchen Li, Shaohui Lin, Zhou Yu
This paper provides a theoretical account of memorization in stochastic interpolation models. By leveraging closed-form expressions for the optimal velocity field and the associate…
RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought
Junbo Qiao, Miaomiao Cai, Wei Li +5
Real-World Image Super-Resolution is one of the most challenging task in image restoration. However, existing methods struggle with an accurate understanding of degraded image cont…
LLaVA-RadZ: Can Multimodal Large Language Models Effectively Tackle Zero-shot Radiology Recognition?
Bangyan Li, Wenxuan Huang, Zhenkun Gao +8
Recently, Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in visual understanding and reasoning across various vision-language tasks. However, w…
DCS-RISR: Dynamic Channel Splitting for Efficient Real-world Image Super-Resolution
Junbo Qiao, Shaohui Lin, Yunlun Zhang +5
Real-world image super-resolution (RISR) has received increased focus for improving the quality of SR images under unknown complex degradation. Existing methods rely on the heavy S…
DF-LLaVA: Unlocking MLLMs for Synthetic Image Detection via Knowledge Injection and Conflict-Driven Self-Reflection
Zhuokang Shen, Kaisen Zhang, Bohan Jia +4
With the increasing prevalence of synthetic images, evaluating image authenticity and locating forgeries accurately while maintaining human interpretability remains a challenging t…
Neural network compression via learnable wavelet transforms
Moritz Wolter, Shaohui Lin, Angela Yao
Wavelets are well known for data compression, yet have rarely been applied to the compression of neural networks. This paper shows how the fast wavelet transform can be used to com…
AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning
Runqi Wang, Xiaoyue Duan, Guoliang Kang +5
Continual learning aims to enable a model to incrementally learn knowledge from sequentially arrived data. Previous works adopt the conventional classification architecture, which…
Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracy
Yunhang Shen, Chaoyou Fu, Shaoqi Dong +14
We introduce Long-VITA, a simple yet effective large multi-modal model for long-context visual-language understanding tasks. It is adept at concurrently processing and analyzing mo…
Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
Xiaoyu Zhan, Wenxuan Huang, Hao Sun +11
Recent advances in Multimodal Large Language Models (MLLMs) have significantly improved 2D visual understanding, prompting interest in their application to complex 3D reasoning tas…
Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution
Simiao Li, Yun Zhang, Wei Li +5
Knowledge distillation (KD) is a promising yet challenging model compression technique that transfers rich learning representations from a well-performing but cumbersome teacher mo…
Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression
Yuntian Tang, Bohan Jia, Wenxuan Huang +7
Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs), yet it incurs substantial computational overhead for inference. E…
Allo{SR}: Rectifying One-Step Super-Resolution to Stay Real via Allomorphic Generative Flows
Zihan Wang, Xudong Huang, Junbo Qiao +4
Real-world image super-resolution (Real-SR) has been revolutionized by leveraging the powerful generative priors from Diffusion Models (DMs) and Flow Matching (FM). However, existi…
WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detection
Haodong Zhu, Wenhao Dong, Linlin Yang +10
Leveraging the complementary characteristics of visible (RGB) and infrared (IR) imagery offers significant potential for improving object detection. In this paper, we propose WaveM…
PAMS: Quantized Super-Resolution via Parameterized Max Scale
Huixia Li, Chenqian Yan, Shaohui Lin +5
Deep convolutional neural networks (DCNNs) have shown dominant performance in the task of super-resolution (SR). However, their heavy memory cost and computation overhead significa…
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
Wenxuan Huang, Bohan Jia, Zijie Zhai +7
DeepSeek-R1-Zero has successfully demonstrated the emergence of reasoning capabilities in LLMs purely through Reinforcement Learning (RL). Inspired by this breakthrough, we explore…
DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning
Yuting Gao, Jia-Xin Zhuang, Shaohui Lin +4
While self-supervised representation learning (SSL) has received widespread attention from the community, recent research argue that its performance will suffer a cliff fall when t…
Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration
Yunshuai Zhou, Junbo Qiao, Jincheng Liao +6
Knowledge distillation (KD) is a valuable yet challenging approach that enhances a compact student network by learning from a high-performance but cumbersome teacher model. However…
Fusion-Mamba for Cross-modality Object Detection
Wenhao Dong, Haodong Zhu, Shaohui Lin +6
Cross-modality fusing complementary information from different modalities effectively improves object detection performance, making it more useful and robust for a wider range of a…
Probability-density-aware Semi-supervised Learning
Shuyang Liu, Ruiqiu Zheng, Yunhang Shen +4
Semi-supervised learning (SSL) assumes that neighbor points lie in the same category (neighbor assumption), and points in different clusters belong to various categories (cluster a…
ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluation
Shizhe Zhou, Bohan Jia, Kai Wu +4
While multimodal large language models (MLLMs) have achieved rapid progress in vision-language understanding, they remain prone to multimodal hallucinations, producing responses th…
Autoregressive Image Generation with Vision Full-view Prompt
Miaomiao Cai, Guanjie Wang, Wei Li +4
In autoregressive (AR) image generation, models based on the 'next-token prediction' paradigm of LLMs have shown comparable performance to diffusion models by reducing inductive bi…
Towards Compact Single Image Super-Resolution via Contrastive Self-distillation
Yanbo Wang, Shaohui Lin, Yanyun Qu +4
Convolutional neural networks (CNNs) are highly successful for super-resolution (SR) but often require sophisticated architectures with heavy memory cost and computational overhead…
TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation
Ling You, Wenxuan Huang, Xinni Xie +5
Soccer is a globally popular sporting event, typically characterized by long matches and distinctive highlight moments. Recent advances in Multimodal Large Language Models (MLLMs)…
Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression
Yuchao Li, Shaohui Lin, Baochang Zhang +5
Compressing convolutional neural networks (CNNs) has received ever-increasing research focus. However, most existing CNN compression methods do not interpret their inherent structu…
Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learning
Zhenwu Shi, Jingyu Gong, Peiwei Wang +7
Text-based human motion editing aims to modify existing motion sequences according to natural language instructions while maintaining the consistency of the original motion. Existi…
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
Shaohui Lin, Rongrong Ji, Chenqian Yan +5
Structured pruning of filters or neurons has received increased focus for compressing convolutional neural networks. Most existing methods rely on multi-stage optimizations in a la…
Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent
Zhen Fang, Yu Zeng, Wenxuan Huang +17
We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding couple…
CLIP-Map: Structured Matrix Mapping for Parameter-Efficient CLIP Compression
Kangjie Zhang, Wenxuan Huang, Xin Zhou +9
Contrastive Language-Image Pre-training (CLIP) has achieved widely applications in various computer vision tasks, e.g., text-to-image generation, Image-Text retrieval and Image cap…
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
Chaoyou Fu, Yuhan Dai, Yongdong Luo +18
In the quest for artificial general intelligence, Multi-modal Large Language Models (MLLMs) have emerged as a focal point in recent advancements. However, the predominant focus rem…
Contrastive Learning for Compact Single Image Dehazing
Haiyan Wu, Yanyun Qu, Shaohui Lin +5
Single image dehazing is a challenging ill-posed problem due to the severe information degeneration. However, existing deep learning based dehazing methods only adopt clear images…