32 citations · 99 across the 23 of their papers we have counts for
10 papers · 1 filter
First-Order LoRA Training with Phone NPUs
Xu Zheng, Wei Cheng, Zhuomin Chen +4
On-device fine-tuning lets a large language model (LLM) learn from its user's private data without uploading it. Low-rank adaptation (LoRA) reduces the trainable parameters, but ea…
Embracing Unknown Step by Step: Towards Reliable Sparse Training in Real World
Bowen Lei, Dongkuan Xu, Ruqi Zhang +1
Sparse training has emerged as a promising method for resource-efficient deep neural networks (DNNs) in real-world applications. However, the reliability of sparse models remains a…
Towards Personalized Federated Learning via Heterogeneous Model Reassembly
Jiaqi Wang, Xingyi Yang, Suhan Cui +4
This paper focuses on addressing the practical yet challenging problem of model heterogeneity in federated learning, where clients possess models with different network structures.…
Time Series Contrastive Learning with Information-Aware Augmentations
Dongsheng Luo, Wei Cheng, Yingheng Wang +8
Various contrastive learning approaches have been proposed in recent years and achieve significant empirical success. While effective and prevalent, contrastive learning has been l…
Calibrating the Rigged Lottery: Making All Tickets Reliable
Bowen Lei, Ruqi Zhang, Dongkuan Xu +1
Although sparse training has been successfully used in various resource-limited deep learning tasks to save memory, accelerate training, and reduce inference time, the reliability…
Efficient Informed Proposals for Discrete Distributions via Newton's Series Approximation
Yue Xiang, Dongyao Zhu, Bowen Lei +2
Gradients have been exploited in proposal distributions to accelerate the convergence of Markov chain Monte Carlo algorithms on discrete distributions. However, these methods requi…
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction
Bowen Lei, Dongkuan Xu, Ruqi Zhang +2
Despite impressive performance, deep neural networks require significant memory and computation costs, prohibiting their application in resource-constrained scenarios. Sparse train…
Accelerating Dataset Distillation via Model Augmentation
Lei Zhang, Jie Zhang, Bowen Lei +6
Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones. Existing DD methods based on gradient…
Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off
Shaoyi Huang, Bowen Lei, Dongkuan Xu +4
Over-parameterization of deep neural networks (DNNs) has shown high prediction accuracy for many applications. Although effective, the large number of parameters hinders its popula…
InfoGCL: Information-Aware Graph Contrastive Learning
Dongkuan Xu, Wei Cheng, Dongsheng Luo +2
Various graph contrastive learning models have been proposed to improve the performance of learning tasks on graph datasets in recent years. While effective and prevalent, these mo…