9 papers
Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation
Run Yang, Runpeng Dai, Jie Sun +5
Sampled-token on-policy distillation (OPD) efficiently transfers capabilities from teacher to student using student-generated tokens, requiring teacher probabilities only for sampl…
GAttNHP: Group Attention Neural Hawkes Process for Extrapolation Reasoning in Temporal Knowledge Graphs
Xiangni Tian, Kaixian Yu, Runpeng Dai +2
Temporal Knowledge Graphs (TKGs) record how facts evolve over time, but forecasting future events on a TKG remains difficult for three reasons: (i) long-range temporal dependencies…
Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling
Runpeng Dai, Tong Zheng, Rui Liu +2
Test-time scaling improves the reasoning performance of large language models but incurs substantial cost in both total computation and latency. Existing adaptive sampling methods…
Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probing
Tong Zheng, Chengsong Huang, Runpeng Dai +9
Parallel thinking has emerged as a promising paradigm for reasoning, yet it imposes significant computational burdens. Existing efficiency methods primarily rely on local, per-traj…
CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models
Runpeng Dai, Linfeng Song, Haolin Liu +8
Reinforcement Learning with Verifiable Rewards (RLVR) is a powerful paradigm for enhancing the reasoning ability of Large Language Models (LLMs). Yet current RLVR methods often exp…
R1-RE: Cross-Domain Relation Extraction with RLVR
Runpeng Dai, Tong Zheng, Run Yang +2
Relation extraction (RE) is a core task in natural language processing. Traditional approaches typically frame RE as a supervised learning problem, directly mapping context to labe…
Deep Distributional Learning with Non-crossing Quantile Network
Guohao Shen, Runpeng Dai, Guojun Wu +3
In this paper, we introduce a non-crossing quantile (NQ) network for conditional distribution learning. By leveraging non-negative activation functions, the NQ network ensures that…
Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models
Runpeng Dai, Run Yang, Fan Zhou +1
Large Language Models (LLMs) and Vision-Language Models (VLMs) have achieved impressive performance across a wide range of tasks, yet they remain vulnerable to carefully crafted pe…
Spatio-temporal Prediction of Fine-Grained Origin-Destination Matrices with Applications in Ridesharing
Run Yang, Runpeng Dai, Siran Gao +3
Accurate spatial-temporal prediction of network-based travelers' requests is crucial for the effective policy design of ridesharing platforms. Having knowledge of the total demand…