4 citations · 4 across the 3 of their papers we have counts for
4 papers
Learning to Revise Reasoning with Segment-wise On-Policy Distillation
Yuxiang Zhang, Ding Cao, Shuting Cui +3
On-policy distillation (OPD) improves large language model reasoning by training students on their own rollouts with dense token-wise supervision from the teacher. However, token-w…
DiaLLMs: EHR Enhanced Clinical Conversational System for Clinical Test Recommendation and Diagnosis Prediction
Weijieying Ren, Tianxiang Zhao, Lei Wang +2
Recent advances in Large Language Models (LLMs) have led to remarkable progresses in medical consultation. However, existing medical LLMs overlook the essential role of Electronic…
Gradient-Aware Logit Adjustment Loss for Long-tailed Classifier
Fan Zhang, Wei Qin, Weijieying Ren +3
In the real-world setting, data often follows a long-tailed distribution, where head classes contain significantly more training samples than tail classes. Consequently, models tra…
Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
Weijieying Ren, Xinlong Li, Lei Wang +2
Existing research has shown that large language models (LLMs) exhibit remarkable performance in language understanding and generation. However, when LLMs are continuously fine-tune…