339 citations · 503 across the 14 of their papers we have counts for
16 papers
Large Language Continuous Diffusion Models
Zhihan Yang, Wei Guo, Jean-Marie Lemercier +14
Despite the success of discrete diffusion language models (dLMs) for fast parallel decoding, their non-smooth, high-dimensional space hinders trajectory steering for reasoning and…
Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse
Taekyung Heo, Rasoul Shafipour, Ritchie Zhao +6
Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver…
CASCADE: Context-Aware Relaxation for Speculative Image Decoding
Selin Yildirim, Subhajit Dutta Chowdhury, Mohammad Mahdi Kamani +2
Autoregressive generation is a powerful approach for high-fidelity image synthesis, but it remains computationally demanding and slow even on the most advanced accelerators. While…
ECCENTRIC: Edge-Cloud Collaboration Framework for Distributed Inference Using Knowledge Adaptation
Mohammad Mahdi Kamani, Zhongwei Cheng, Lin Chen
The massive growth in the utilization of edge AI has made the applications of machine learning models ubiquitous in different domains. Despite the computation and communication eff…
Distributed Personalized Empirical Risk Minimization
Yuyang Deng, Mohammad Mahdi Kamani, Pouria Mahdavinia +1
This paper advocates a new paradigm Personalized Empirical Risk Minimization (PERM) to facilitate learning from heterogeneous data sources without imposing stringent constraints on…
FedRule: Federated Rule Recommendation System with Graph Neural Networks
Yuhang Yao, Mohammad Mahdi Kamani, Zhongwei Cheng +3
Much of the value that IoT (Internet-of-Things) devices bring to ``smart'' homes lies in their ability to automatically trigger other devices' actions: for example, a smart camera…
Learning Distributionally Robust Models at Scale via Composite Optimization
Farzin Haddadpour, Mohammad Mahdi Kamani, Mehrdad Mahdavi +1
To train machine learning models that are robust to distribution shifts in the data, distributionally robust optimization (DRO) has been proven very effective. However, the existin…
Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation
Sumanth Chennupati, Mohammad Mahdi Kamani, Zhongwei Cheng +1
Knowledge Distillation is becoming one of the primary trends among neural network compression algorithms to improve the generalization performance of a smaller student model with g…
Local SGD Optimizes Overparameterized Neural Networks in Polynomial Time
Yuyang Deng, Mohammad Mahdi Kamani, Mehrdad Mahdavi
In this paper we prove that Local (S)GD (or FedAvg) can optimize deep neural networks with Rectified Linear Unit (ReLU) activation function in polynomial time. Despite the establis…
Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing
Mohammad Mahdi Kamani, Rana Forsati, James Z. Wang +1
As algorithmic decision-making systems are becoming more pervasive, it is crucial to ensure such systems do not become mechanisms of unfair discrimination on the basis of gender, r…
Distributionally Robust Federated Averaging
Yuyang Deng, Mohammad Mahdi Kamani, Mehrdad Mahdavi
In this paper, we study communication efficient distributed algorithms for distributionally robust federated learning via periodic averaging with adaptive sampling. In contrast to…
Federated Learning with Compression: Unified Analysis and Sharp Guarantees
Farzin Haddadpour, Mohammad Mahdi Kamani, Aryan Mokhtari +1
In federated learning, communication cost is often a critical bottleneck to scale up distributed optimization algorithms to collaboratively learn a model from millions of devices w…
Adaptive Personalized Federated Learning
Yuyang Deng, Mohammad Mahdi Kamani, Mehrdad Mahdavi
Investigation of the degree of personalization in federated learning algorithms has shown that only maximizing the performance of the global model will confine the capacity of the…
Efficient Fair Principal Component Analysis
Mohammad Mahdi Kamani, Farzin Haddadpour, Rana Forsati +1
It has been shown that dimension reduction methods such as PCA may be inherently prone to unfairness and treat data from different sensitive groups such as race, color, sex, etc.,…
Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization
Farzin Haddadpour, Mohammad Mahdi Kamani, Mehrdad Mahdavi +1
Communication overhead is one of the key challenges that hinders the scalability of distributed optimization algorithms. In this paper, we study local distributed SGD, where data i…
CAPTAIN: Comprehensive Composition Assistance for Photo Taking
Farshid Farhat, Mohammad Mahdi Kamani, James Z. Wang
Many people are interested in taking astonishing photos and sharing with others. Emerging hightech hardware and software facilitate ubiquitousness and functionality of digital phot…