Shayan Mohajer Hamidi

Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models

Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin +7

Mixture-of-Experts (MoE) models reduce computation through sparse expert activation, but deployment still requires storing a large number of expert parameters. We study post-traini…

cs.AI2026

Budgeted KV-Cache Repair for Evolving Contexts

Han Yu, Wenhui Zhu, Jincheng Cao +15

Language-model agents revise their working context as information becomes obsolete, but deleting text does not erase its influence on cached computation. Later key-value (KV) state…

math.CV2026

Maximum-modulus points on concentric circles

Yunpeng Li

We give a negative answer to the second question of Erdős Problem #1117: can the number $ν_{f} (r)$ of maximum-modulus points of a nonzero, non-monomial entire function on $∣ z ∣ = r$ ten…

Composing What Each Teacher Learned: Delta Distillation for Multi-Teacher On-Policy Learning

Hejian Sang, Yuanda Xu, Zhengze Zhou +4

Multi-teacher on-policy distillation (MOPD) is used in two settings. In common-domain composition, several teachers score each student rollout from one prompt domain and their sign…

cs.AI2026

Hierarchical Reinforcement Learning with Stable Temporal Abstraction for Language Model Agents

Shayan Mohajer Hamidi, Yize Cheng, Yuanda Xu +2

Hierarchical reinforcement learning improves long-horizon control by organizing primitive actions around persistent subgoals and assigning credit at multiple temporal scales. Recen…

Smaller Models, Better Rejects: Preference Distillation Scaling

Rui Cai, Wenhui Zhu, Xiwen Chen +15

Preference distillation typically treats a teacher response as preferred and the student's own response as rejected. This assumes that self-generated failures are the most informat…

Regime-aware financial volatility forecasting via in-context learning

Saba Asaad, Shayan Mohajer Hamidi, Ali Bereyhi

This work introduces a regime-aware in-context learning framework that leverages large language models (LLMs) for financial volatility forecasting under nonstationary market condit…

cs.CV2026

ASMIL: Attention-Stabilized Multiple Instance Learning for Whole Slide Imaging

Linfeng Ye, Shayan Mohajer Hamidi, Zhixiang Chi +5

Attention-based multiple instance learning (MIL) has emerged as a powerful framework for whole slide image (WSI) diagnosis, leveraging attention to aggregate instance-level feature…

Coupled Data and Measurement Space Dynamics for Enhanced Diffusion Posterior Sampling

Shayan Mohajer Hamidi, En-Hui Yang, Ben Liang

Inverse problems, where the goal is to recover an unknown signal from noisy or incomplete measurements, are central to applications in medical imaging, remote sensing, and computat…

Information-Guided Diffusion Sampling for Dataset Distillation

Linfeng Ye, Shayan Mohajer Hamidi, Guang Li +3

Dataset distillation aims to create a compact dataset that retains essential information while maintaining model performance. Diffusion models (DMs) have shown promise for this tas…

Towards Undistillable Models by Minimizing Conditional Mutual Information

Linfeng Ye, Shayan Mohajer Hamidi, En-hui Yang

A deep neural network (DNN) is said to be undistillable if, when used as a black-box input-output teacher, it cannot be distilled through knowledge distillation (KD). In this case,…

Distributed Quasi-Newton Method for Fair and Fast Federated Learning

Shayan Mohajer Hamidi, Linfeng Ye

Federated learning (FL) is a promising technology that enables edge devices/clients to collaboratively and iteratively train a machine learning model under the coordination of a ce…

Coded Deep Learning: Framework and Algorithm

En-hui Yang, Shayan Mohajer Hamidi

The success of deep learning (DL) is often achieved with large models and high complexity during both training and post-training inferences, hindering training in resource-limited…

Conditional Mutual Information Based Diffusion Posterior Sampling for Solving Inverse Problems

Shayan Mohajer Hamidi, En-Hui Yang

Inverse problems are prevalent across various disciplines in science and engineering. In the field of computer vision, tasks such as inpainting, deblurring, and super-resolution ar…

cs.CV2024

Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling

Shayan Mohajer Hamidi, En-Hui Yang

Inverse problems exist in many disciplines of science and engineering. In computer vision, for example, tasks such as inpainting, deblurring, and super resolution can be effectivel…

GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learning

Shayan Mohajer Hamidi, Ali Bereyhi, Saba Asaad +1

Second-order methods are widely adopted to improve the convergence rate of learning algorithms. In federated learning (FL), these methods require the clients to share their local H…

Rate-Constrained Quantization for Communication-Efficient Federated Learning

Shayan Mohajer Hamidi, Ali Bereyhi

Quantization is a common approach to mitigate the communication cost of federated learning (FL). In practice, the quantized local parameters are further encoded via an entropy codi…

How to Train the Teacher Model for Effective Knowledge Distillation

Shayan Mohajer Hamidi, Xizhen Deng, Renhao Tan +2

Recently, it was shown that the role of the teacher in knowledge distillation (KD) is to provide the student with an estimate of the true Bayes conditional probability density (BCP…

Adversarial Training via Adaptive Knowledge Amalgamation of an Ensemble of Teachers

Shayan Mohajer Hamidi, Linfeng Ye

Adversarial training (AT) is a popular method for training robust deep neural networks (DNNs) against adversarial attacks. Yet, AT suffers from two shortcomings: (i) the robustness…