66 citations · 132 across the 26 of their papers we have counts for
9 papers · 1 filter
Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
Xinmu Ge, Zizhuo Zhang, Yu Huang +9
On-policy distillation (OPD) has emerged as a promising post-training technique for enhancing LLM reasoning. Under the reverse KL objective, the idealized optimum of OPD aligns the…
IDER: IDempotent Experience Replay for Reliable Continual Learning
Zhanwang Liu, Yuting Li, Haoyuan Gao +4
Catastrophic forgetting, the tendency of neural networks to forget previously learned knowledge when learning new tasks, has been a major challenge in continual learning (CL). To t…
PANTHER: Generative Pretraining Beyond Language for Sequential User Behavior Modeling
Guilin Li, Yun Zhang, Xiuyuan Chen +6
Large language models (LLMs) have shown that generative pretraining can distill vast world knowledge into compact token representations. While LLMs encapsulate extensive world know…
EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework
Chen Wang, Lai Wei, Yanzhi Zhang +5
Recent advances in reinforcement learning (RL) have significantly enhanced the reasoning capabilities of large language models (LLMs). Group Relative Policy Optimization (GRPO), a…
SAFE: Slow and Fast Parameter-Efficient Tuning for Continual Learning with Pre-Trained Models
Linglan Zhao, Xuerui Zhang, Ke Yan +2
Continual learning aims to incrementally acquire new concepts in data streams while resisting forgetting previous knowledge. With the rise of powerful pre-trained models (PTMs), th…
Investigating the Impact of Model Complexity in Large Language Models
Jing Luo, Huiyuan Wang, Weiran Huang
Large Language Models (LLMs) based on the pre-trained fine-tuning paradigm have become pivotal in solving natural language processing tasks, consistently achieving state-of-the-art…
A Statistical Theory of Regularization-Based Continual Learning
Xuyang Zhao, Huiyuan Wang, Weiran Huang +1
We provide a statistical analysis of regularization-based continual learning on a sequence of linear regression tasks, with emphasis on how different regularization terms affect th…
Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
Fuli Feng, Weiran Huang, Xiangnan He +3
Graph Convolutional Network (GCN) is an emerging technique for information retrieval (IR) applications. While GCN assumes the homophily property of a graph, real-world graphs are n…
GraphAIR: Graph Representation Learning with Neighborhood Aggregation and Interaction
Fenyu Hu, Yanqiao Zhu, Shu Wu +3
Graph representation learning is of paramount importance for a variety of graph analytical tasks, ranging from node classification to community detection. Recently, graph convoluti…