936 citations · 2.4k across the 134 of their papers we have counts for
4 papers · 2 filters
An Information Theory of Compute-Optimal Size Scaling, Emergence, and Plateaus in Language Models
Anuj K. Nayak, Lav R. Varshney
Recent empirical studies show three phenomena with increasing size of language models: compute-optimal size scaling, emergent capabilities, and performance plateauing. We present a…
Compute-Update Federated Learning: A Lattice Coding Approach Over-the-Air
Seyed Mohammad Azimi-Abarghouyi, Lav R. Varshney
This paper introduces a federated learning framework that enables over-the-air computation via digital communications, using a new joint source-channel coding scheme. Without relyi…
Semantic Compression with Information Lattice Learning
Haizi Yu, Lav R. Varshney
Data-driven artificial intelligence (AI) techniques are becoming prominent for learning in support of data compression, but are focused on standard problems such as text compressio…
Federated Learning via Lattice Joint Source-Channel Coding
Seyed Mohammad Azimi-Abarghouyi, Lav R. Varshney
This paper introduces a universal federated learning framework that enables over-the-air computation via digital communications, using a new joint source-channel coding scheme. Wit…