3 citations · 7 across the 16 of their papers we have counts for
7 papers · 1 filter
CST: Collaborative Selective Transmission for Communication-Efficient Multimodal Edge Inference
Hai Chi, Junrui Zhang, Rui Ning +4
Collaborative multimodal inference improves edge perception by combining observations from distributed sensing devices, but transmitting high-dimensional helper representations inc…
GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
Jianing Deng, Song Wang, Dongwei Wang +4
Mixture-of-Experts Large Language Models (MoE-LLMs) achieve strong performance but incur substantial memory overhead due to massive expert parameters. Mixed-precision quantization…
From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation
Yuxin Ren, Maxwell D Collins, Miao Hu +1
Self-attention serves as the core foundation of large-scale transformer pretraining, but its quadratic token interaction cost makes inference expensive. Replacing attention with si…
BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models
Liulu He, Shenli Zheng, Karwei Sun +6
Rotations have become essential to state-of-the-art quantization pipelines for large language models (LLMs) by effectively smoothing outliers in weights and activations. However, f…
PAT: Pruning-Aware Tuning for Large Language Models
Yijiang Liu, Huanrui Yang, Youxin Chen +4
Large language models (LLMs) excel in language tasks, especially with supervised fine-tuning after pre-training. However, their substantial memory and computational requirements hi…
T-REX: Mixture-of-Rank-One-Experts with Semantic-aware Intuition for Multi-task Large Language Model Finetuning
Rongyu Zhang, Yijiang Liu, Huanrui Yang +5
Large language models (LLMs) encounter significant adaptation challenges in diverse multitask finetuning. Mixture-of-experts (MoE) provides a promising solution with a dynamic arch…
Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting
Anthony Chen, Huanrui Yang, Yulu Gan +7
Uncertainty estimation is crucial for machine learning models to detect out-of-distribution (OOD) inputs. However, the conventional discriminative deep learning classifiers produce…