3 citations · 3 across the 9 of their papers we have counts for
5 papers · 1 filter
Reimagining Parameter Space Exploration with Diffusion Models
Lijun Zhang, Xiao Liu, Hui Guan
Adapting neural networks to new tasks typically requires task-specific fine-tuning, which is time-consuming and reliant on labeled data. We explore a generative alternative that pr…
Memory Savings at What Cost? A Study of Alternatives to Backpropagation
Kunjal Panchal, Sunav Choudhary, Yuriy Brun +1
Forward-mode automatic differentiation (FmAD) and zero-order (ZO) optimization are increasingly proposed as memory-efficient, backpropagation-free alternatives for large language m…
ASTRA: Communication-Efficient Acceleration for Multi-Device Transformer Inference
Xiao Liu, Lijun Zhang, Deepak Ganesan +1
Multi-device inference can reduce Transformer latency by parallelizing computation. However, existing methods require high inter-device bandwidth, making them impractical for bandw…
Assessing Simplification Levels in Neural Networks: The Impact of Hyperparameter Configurations on Complexity and Sensitivity
Huixin Guan
This paper presents an experimental study focused on understanding the simplification properties of neural networks under different hyperparameter configurations, specifically inve…
Thinking Forward: Memory-Efficient Federated Finetuning of Language Models
Kunjal Panchal, Nisarg Parikh, Sunav Choudhary +3
Finetuning large language models (LLMs) in federated learning (FL) settings has become increasingly important as it allows resource-constrained devices to finetune a model using pr…