5 papers
Distributed Perceptron under Bounded Staleness, Partial Participation, and Noisy Communication
Keval Jain, Anant Raj, Saurav Prakash +1
We study a semi-asynchronous client-server perceptron trained via iterative parameter mixing (IPM-style averaging): clients run local perceptron updates and a server forms a global…
Federated Nonlinear System Identification
Omkar Tupe, Max Hartman, Lav R. Varshney +1
We consider federated learning of linearly-parameterized nonlinear systems. We establish theoretical guarantees on the effectiveness of federated nonlinear system identification co…
Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference
Nitin Priyadarshini Shankar, Soham Lahiri, Sheetal Kalyani +1
Federated Learning (FL) preserves privacy by distributing training across devices. However, using DNNs is computationally intensive at the low-powered edge during inference. Edge d…
SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
Santhosh G S, Saurav Prakash, Balaraman Ravindran
Large Language Models (LLMs) face a significant bottleneck during autoregressive inference due to the massive memory footprint of the Key-Value (KV) cache. Existing compression tec…
AQUA: Attention via QUery mAgnitudes for Memory and Compute Efficient Inference in LLMs
Santhosh G S, Saurav Prakash, Balaraman Ravindran
The quadratic complexity of the attention mechanism remains a fundamental barrier to scaling Large Language Models (LLMs) to longer contexts, creating a critical bottleneck in both…