10 papers
CoRD: Coupled Rollouts Fill the Blind Spot of On-Policy Distillation
Guhan Chen, Bohan Li, Jun Rao +8
On-policy distillation trains a student on its own rollouts, so states the teacher would reach but the student does not are seldom supervised. We introduce CoRD (coupled-rollout di…
DIAG: Diagnostic Iterative Alignment and Generation for Data-Efficient Mathematical Preference Distillation
Guhan Chen, Songtao Tian, Bohan Li +3
Iterative preference optimization is essential for aligning Large Language Models on mathematical reasoning tasks, yet its efficiency is often throttled by signal scarcity: as the…
Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learning
Songtao Tian, Guhan Chen, Bohan Li +2
Consistency distillation has significantly accelerated diffusion-model inference, but its sampling dynamics remain underexplored. We reveal an asymmetry: although Logit-Normal samp…
Divergence of Empirical Neural Tangent Kernel in Classification Problems
Zixiong Yu, Songtao Tian, Guhan Chen
This paper demonstrates that in classification problems, fully connected neural networks (FCNs) and residual neural networks (ResNets) cannot be approximated by kernel logistic reg…
Branch Scaling Manifests as Implicit Architectural Regularization for Improving Generalization in Overparameterized ResNets
Zixiong Yu, Guhan Chen, Jianfa Lai +2
Scaling factors in residual branches have emerged as a prevalent method for boosting neural network performance, especially in normalization-free architectures. While prior work ha…
Functional Slicing-free Inverse Regression via Martingale Difference Divergence Operator
Songtao Tian, Zixiong Yu, Rui Chen
Functional sliced inverse regression (FSIR) is one of the most popular algorithms for functional sufficient dimension reduction (FSDR). However, the choice of slice scheme in FSIR…
On the Optimality of Functional Sliced Inverse Regression
Rui Chen, Songtao Tian, Dongming Huang +2
In this paper, we prove that functional sliced inverse regression (FSIR) achieves the optimal (minimax) rate for estimating the central space in functional sufficient dimension red…
Generalization Ability of Wide Residual Networks
Jianfa Lai, Zixiong Yu, Songtao Tian +1
In this paper, we study the generalization ability of the wide residual network on with the ReLU activation function. We first show that as the width $m\rightarr…
On the Structural Dimension of Sliced Inverse Regression
Dongming Huang, Songtao Tian, Qian Lin
In this work, we address the longstanding puzzle that Sliced Inverse Regression (SIR) often performs poorly for sufficient dimension reduction when the structural dimension (th…
Equivalence of state equations from different methods in High-dimensional Regression
Saidi Luo, Songtao Tian
State equations (SEs) were firstly introduced in the approximate message passing (AMP) to describe the mean square error (MSE) in compressed sensing. Since then a set of state equa…