3 citations · 7 across the 16 of their papers we have counts for
17 papers
CST: Collaborative Selective Transmission for Communication-Efficient Multimodal Edge Inference
Hai Chi, Junrui Zhang, Rui Ning +4
Collaborative multimodal inference improves edge perception by combining observations from distributed sensing devices, but transmitting high-dimensional helper representations inc…
SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers
Jianing Deng, Yuanzhe Li, Jialu Wang +4
Feed-forward 3D reconstruction (F3R) transformers have recently achieved remarkable success. However, scaling them to long image sequences remains challenging, as the quadratic com…
GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
Jianing Deng, Song Wang, Dongwei Wang +4
Mixture-of-Experts Large Language Models (MoE-LLMs) achieve strong performance but incur substantial memory overhead due to massive expert parameters. Mixed-precision quantization…
MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation
Ronyu Zhang, Aosong Cheng, Gaole Dai +8
Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams while retaining past competence, yet texture-biased backbones risk error…
From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation
Yuxin Ren, Maxwell D Collins, Miao Hu +1
Self-attention serves as the core foundation of large-scale transformer pretraining, but its quadratic token interaction cost makes inference expensive. Replacing attention with si…
SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models
Hengyu Fang, Yijiang Liu, Yuan Du +2
Vision-Language-Action (VLA) models exhibit unprecedented capabilities for embodied intelligence. However, their extensive computational and memory costs hinder their practical dep…
BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models
Liulu He, Shenli Zheng, Karwei Sun +6
Rotations have become essential to state-of-the-art quantization pipelines for large language models (LLMs) by effectively smoothing outliers in weights and activations. However, f…
FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models
Zhongyu Zhao, Menghang Dong, Rongyu Zhang +6
Recent research has demonstrated that Feed-Forward Networks (FFNs) in Large Language Models (LLMs) play a pivotal role in storing diverse linguistic and factual knowledge. Conventi…
PAT: Pruning-Aware Tuning for Large Language Models
Yijiang Liu, Huanrui Yang, Youxin Chen +4
Large language models (LLMs) excel in language tasks, especially with supervised fine-tuning after pre-training. However, their substantial memory and computational requirements hi…
Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
Haiquan Lu, Xiaotian Liu, Yefan Zhou +6
Recent studies on deep ensembles have identified the sharpness of the local minima of individual learners and the diversity of the ensemble members as key factors in improving test…
Fisher-aware Quantization for DETR Detectors with Critical-category Objectives
Huanrui Yang, Yafeng Huang, Zhen Dong +6
The impact of quantization on the overall performance of deep learning models is a well-studied problem. However, understanding and mitigating its effects on a more fine-grained le…
Decomposing the Neurons: Activation Sparsity via Mixture of Experts for Continual Test Time Adaptation
Rongyu Zhang, Aosong Cheng, Yulin Luo +8
Continual Test-Time Adaptation (CTTA), which aims to adapt the pre-trained model to ever-evolving target domains, emerges as an important task for vision models. As current vision…
T-REX: Mixture-of-Rank-One-Experts with Semantic-aware Intuition for Multi-task Large Language Model Finetuning
Rongyu Zhang, Yijiang Liu, Huanrui Yang +5
Large language models (LLMs) encounter significant adaptation challenges in diverse multitask finetuning. Mixture-of-experts (MoE) provides a promising solution with a dynamic arch…
Magic-Me: Identity-Specific Video Customized Diffusion
Ze Ma, Daquan Zhou, Chun-Hsiao Yeh +6
Creating content with specified identities (ID) has attracted significant interest in the field of generative models. In the field of text-to-image generation (T2I), subject-driven…
VeCAF: Vision-language Collaborative Active Finetuning with Training Objective Awareness
Rongyu Zhang, Zefan Cai, Huanrui Yang +9
Finetuning a pretrained vision model (PVM) is a common technique for learning downstream vision tasks. However, the conventional finetuning process with randomly sampled data point…
Efficient Deweather Mixture-of-Experts with Uncertainty-aware Feature-wise Linear Modulation
Rongyu Zhang, Yulin Luo, Jiaming Liu +8
The Mixture-of-Experts (MoE) approach has demonstrated outstanding scalability in multi-task learning including low-level upstream tasks such as concurrent removal of multiple adve…
Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting
Anthony Chen, Huanrui Yang, Yulu Gan +7
Uncertainty estimation is crucial for machine learning models to detect out-of-distribution (OOD) inputs. However, the conventional discriminative deep learning classifiers produce…