11 citations · 16 across the 16 of their papers we have counts for
8 papers · 1 filter
CRIP: Channel Level Representation Injection for Personalized One-Shot Federated Learning
Zijian Jiang, Chaoli Sun, Handing Wang +1
One-shot federated learning (OSFL) has emerged as a promising collaborative model learning framework with only a single round of communication, offering significant advantages in c…
BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding
Liang He, Jingbo Wen, Qishi Zhan +4
Speculative decoding speeds up autoregressive decoding by using a drafter to propose multiple tokens that a verifier validates in parallel. In resource-constrained deployments, the…
ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
Jiaxi Li, Lu Yin, Li Shen +5
Large Language Models (LLMs) have achieved remarkable capabilities, but their immense computational demands during training remain a critical bottleneck for widespread adoption. Lo…
Long Chain-of-Thought Compression via Fine-Grained Group Policy Optimization
Xinchen Han, Hossam Afifi, Michel Marot +2
Large Language Models (LLMs) often generate unnecessarily verbose Chain-of-Thought (CoT) reasoning that increases computational costs and latency without proportional performance g…
LOST: Low-rank and Sparse Pre-training for Large Language Models
Jiaxi Li, Lu Yin, Li Shen +6
While large language models (LLMs) have achieved remarkable performance across a wide range of tasks, their massive scale incurs prohibitive computational and memory costs for pre-…
OWLed: Outlier-weighed Layerwise Pruning for Efficient Autonomous Driving Framework
Jiaxi Li, Lu Yin, Xilu Wang
The integration of Large Language Models (LLMs) into autonomous driving systems offers promising enhancements in environmental understanding and decision-making. However, the subst…
MO-EMT-NAS: Multi-Objective Continuous Transfer of Architectural Knowledge Between Tasks from Different Datasets
Peng Liao, XiLu Wang, Yaochu Jin +1
Deploying models across diverse devices demands tradeoffs among multiple objectives due to different resource constraints. Arguably, due to the small model trap problem in multi-ob…
Recent Advances in Bayesian Optimization
Xilu Wang, Yaochu Jin, Sebastian Schmitt +1
Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the dev…