16 citations · 26 across the 7 of their papers we have counts for
11 papers
Reward-Aligned Distillation
Yilan Chen, Zian Su, Manish Prajapat +4
Reinforcement learning with verifiable rewards improves the reasoning capabilities of large language models, but remains computationally expensive and sample inefficient. On-policy…
Token Utility for On-Policy Distillation via Local Trust-Region Optimization
Narayanan Sadagopan, Yilan Chen, Vijay Lingam
Token-selection methods for on-policy distillation (OPD) prioritize student-visited positions using diverse signals, including uncertainty, teacher-student disagreement, and mismat…
BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving
Seth Z. Zhao, Luobin Wang, Hongwei Ruan +13
Open-loop (OL) to closed-loop (CL) gap (OL-CL gap) exists when OL-pretrained policies scoring high in OL evaluations fail to transfer effectively in closed-loop (CL) deployment. In…
How Does Label Noise Gradient Descent Improve Generalization in the Low SNR Regime?
Wei Huang, Andi Han, Yujin Song +4
The capacity of deep learning models is often large enough to both learn the underlying statistical signal and overfit to noise in the training set. This noise memorization can be…
Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel
Yilan Chen, Zhichao Wang, Wei Huang +3
Gradient-based optimization methods have shown remarkable empirical success, yet their theoretical generalization properties remain only partially understood. In this paper, we est…
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm
Zhanpeng Zhou, Zijun Chen, Yilan Chen +2
The pretraining-finetuning paradigm has become the prevailing trend in modern deep learning. In this work, we discover an intriguing linear phenomenon in models that are initialize…
The Importance of Prompt Tuning for Automated Neuron Explanations
Justin Lee, Tuomas Oikarinen, Arjun Chatha +3
Recent advances have greatly increased the capabilities of large language models (LLMs), but our understanding of the models and their safety has not progressed as fast. In this pa…
Quantifying the Knowledge in a DNN to Explain Knowledge Distillation for Classification
Quanshi Zhang, Xu Cheng, Yilan Chen +1
Compared to traditional learning from scratch, knowledge distillation sometimes makes the DNN achieve superior performance. This paper provides a new perspective to explain the suc…
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning
Wei Huang, Chunrui Liu, Yilan Chen +2
PAC-Bayesian is an analysis framework where the training error can be expressed as the weighted average of the hypotheses in the posterior distribution whilst incorporating the pri…
On the Equivalence between Neural Network and Support Vector Machine
Yilan Chen, Wei Huang, Lam M. Nguyen +1
Recent research shows that the dynamics of an infinitely wide neural network (NN) trained by gradient descent can be characterized by Neural Tangent Kernel (NTK) \citep{jacot2018ne…
Explaining Knowledge Distillation by Quantifying the Knowledge
Xu Cheng, Zhefan Rao, Yilan Chen +1
This paper presents a method to interpret the success of knowledge distillation by quantifying and analyzing task-relevant and task-irrelevant visual concepts that are encoded in i…