9 citations · 29 across the 22 of their papers we have counts for
10 papers · 1 filter
CoMem: Context Management with A Decoupled Long-Context Model
Yuwei Zhang, Chengyu Dong, Shuowei Jin +11
Context management enables agentic models to solve long-horizon tasks through iterative summarization of previous interaction histories. However, this process typically incurs subs…
Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation
Yuwei Zhang, Sha Li, Changlong Yu +9
Enabling Large Language Models (LLMs) to continuously improve from environmental interactions is a central challenge in post-training. While on-policy self-distillation offers a pr…
ReMem: Mutual Information-Aware Fine-tuning of Pretrained Vision Transformers for Effective Knowledge Distillation
Chengyu Dong, Huan Gui, Noveen Sachdeva +6
Knowledge distillation from pretrained visual representation models offers an effective approach to improve small, task-specific production models. However, the effectiveness of su…
Physics-Informed Data Denoising for Real-Life Sensing Systems
Xiyuan Zhang, Xiaohan Fu, Diyan Teng +9
Sensors measuring real-life physical processes are ubiquitous in today's interconnected world. These sensors inherently bear noise that often adversely affects performance and reli…
Bridging Discrete and Backpropagation: Straight-Through and Beyond
Liyuan Liu, Chengyu Dong, Xiaodong Liu +2
Backpropagation, the cornerstone of deep learning, is limited to computing gradients for continuous variables. This limitation poses challenges for problems involving discrete late…
Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications
Chengyu Dong
Deep neural networks have seen enormous success in various real-world applications. Beyond their predictions as point estimates, increasing attention has been focused on quantifyin…
Toward Student-Oriented Teacher Network Training For Knowledge Distillation
Chengyu Dong, Liyuan Liu, Jingbo Shang
How to conduct teacher training for knowledge distillation is still an open problem. It has been widely observed that a best-performing teacher does not necessarily yield the best-…
Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting
Chengyu Dong, Liyuan Liu, Jingbo Shang
We show that label noise exists in adversarial training. Such label noise is due to the mismatch between the true label distribution of adversarial examples and the label inherited…
Data Quality Matters For Adversarial Training: An Empirical Study
Chengyu Dong, Liyuan Liu, Jingbo Shang
Multiple intriguing problems are hovering in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off. These problems pose g…
Overfitting or Underfitting? Understand Robustness Drop in Adversarial Training
Zichao Li, Liyuan Liu, Chengyu Dong +1
Our goal is to understand why the robustness drops after conducting adversarial training for too long. Although this phenomenon is commonly explained as overfitting, our analysis s…