4 papers
RELAY: Selective Self-Distillation with Adaptive Hand-off for Rubric-Based Reinforcement Learning
Kunwang Zheng, Xinyu Sun, Yu Cao +7
Rubric-based reinforcement learning grades a response against many written criteria but collapses them into one reward, which discards the detail the rubric carries and gives no GR…
Performance Monitoring of Proton Exchange Membrane Water Electrolyzer by Transformers-Based Machine Learning Model
Bingqing Chen, Ivan Batalov, Qiu Chen +2
Green hydrogen plays an essential role in decarbonization, with capacity projected to scale to 560 GW by 2030 (vs. 1.39 GW in 2023) in net-zero settings. Proton exchange membrane (…
Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective
Junpeng Zhang, Lei Cheng, Guoxi Zhang +3
This paper explores a scientific question in supervised fine-tuning (SFT): why SFT is broadly effective for small-scale deep neural networks, yet can produce inconsistent or even d…
Self-Distillation Improves DNA Sequence Inference
Tong Yu, Lei Cheng, Ruslan Khalitov +2
Self-supervised pretraining (SSP) has been recognized as a method to enhance prediction accuracy in various downstream tasks. However, its efficacy for DNA sequences remains somewh…