8 papers · 1 filter
Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
Lei Zhao, Qichao Zhao, Bowen Zuo +1
On-policy distillation (OPD) enables effective capability transfer between language models, yet the mechanisms underlying its failures are not fully understood. Across code generat…
Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees
Zihan Dong, Rui Qian, Qishi Zhan +3
Computer-use agents often fail on transient GUI events because they produce the correct action only after the relevant window has already closed. We identify the main cause as expe…
BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding
Liang He, Jingbo Wen, Qishi Zhan +4
Speculative decoding speeds up autoregressive decoding by using a drafter to propose multiple tokens that a verifier validates in parallel. In resource-constrained deployments, the…
Relative Repairability: A Calibration-Based Diagnostic for High-Sparsity Post-Pruning Allocation
Qishi Zhan, Liang He, Minxuan Hu +1
At very high sparsity, neural network pruning does more than decide which weights remain. It also determines where pruning induced damage is placed across the network, and whether…
How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability
Qishi Zhan, Minxuan Hu, Liang He
Unstructured magnitude pruning at high sparsity can reduce neural network accuracy to near-random performance, while labeled retraining may be unavailable in practical deployment s…
Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks
Qishi Zhan, Ziheng Chen, Minxuan Hu
One-shot magnitude pruning can cause severe accuracy collapse in the high-sparsity regime, even when the pruning mask preserves the largest weights. We argue that this failure refl…
A Tale of Two Variances: When Single-Seed Benchmarks Fail in Bayesian Deep Learning
Qishi Zhan, Minxuan Hu, Liang He +2
In limited-data settings, a single endpoint mean of an evaluation metric such as the Continuous Ranked Probability Score (CRPS) is itself a random variable, yet it is routinely rep…
Unstable Rankings in Bayesian Deep Learning Evaluation
Qishi Zhan, Minxuan Hu, Guansu Wang +2
Standard evaluations of Bayesian deep learning methods assume that metric estimates are reliable, but we show this assumption fails under data scarcity. Method rankings are not onl…