35 citations · 81 across the 15 of their papers we have counts for
15 papers
GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization
Chuanyang Zheng, Jiankai Sun, Yihang Gao +11
The placement of normalization layers, specifically Pre-Norm and Post-Norm, remains an open question in Transformer architecture design. In this work, we rethink these approaches t…
Swapped Logit Distillation via Bi-level Teacher Alignment
Stephen Ekaputra Limantoro, Jhe-Hao Lin, Chih-Yu Wang +4
Knowledge distillation (KD) compresses the network capacity by transferring knowledge from a large (teacher) network to a smaller one (student). It has been mainstream that the tea…
MathChat: Converse to Tackle Challenging Math Problems with LLM Agents
Yiran Wu, Feiran Jia, Shaokun Zhang +7
Employing Large Language Models (LLMs) to address mathematical problems is an intriguing research endeavor, considering the abundance of math problems expressed in natural language…
HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shifts
Shaokun Zhang, Yiran Wu, Zhonghua Zheng +2
In this work, we propose a hyperparameter optimization method named \emph{HyperTime} to find hyperparameters robust to potential temporal distribution shifts in the unseen test dat…
Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference
Chi Wang, Susan Xueqing Liu, Ahmed H. Awadallah
Large Language Models (LLMs) have sparked significant interest in their generative capabilities, leading to the development of various commercial applications. The high cost of usi…
ACE: Adaptive Constraint-aware Early Stopping in Hyperparameter Optimization
Yi-Wei Chen, Chi Wang, Amin Saied +1
Deploying machine learning models requires high model quality and needs to comply with application constraints. That motivates hyperparameter optimization (HPO) to tune model confi…