4 papers
Capturing LLM Capabilities via Evidence-Calibrated Query Clustering
Fangzhou Wu, Sandeep Silwal, Qiuyi Zhang
Query clustering organizes queries into groups that reflect shared latent capability demands, enabling capability-aware LLM evaluation. Existing clustering methods, which primarily…
DynMuon: A Dynamic Spectral Shaping View of Muon
Fangzhou Wu, Rikhav Shah, Sandeep Silwal +1
In recent years, Muon has emerged as the dominant method for training large language models, and transformers more broadly. The essential difference, when compared to standard grad…
Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise
Taha El Bakkali, El Mahdi Chayti, Qiuyi Zhang +2
We study stochastic zeroth-order (ZO) optimization of smooth nonconvex objectives under heavy-tailed sample-gradient noise. This regime is motivated by empirical evidence that grad…
Language Model Embeddings Can Be Sufficient for Bayesian Optimization
Tung Nguyen, Qiuyi Zhang, Bangding Yang +6
Bayesian Optimization is ubiquitous in experimental design and black-box optimization for improving search efficiency. However, most existing approaches rely on regression models w…