6 papers
Regret Pre-training: Bridging Prior and Posterior Views for Enhanced Knowledge Grounding
Mingkuan Zhao, Xiayu Sun, Wentao Hu +5
Causal language models factorize sequence probabilities using only preceding context, leaving future information unexploited during training despite its availability in the trainin…
BioDefect: The First Dataset for Defect Detection in Bioinformatics Software
Tianxiang Xu, Xiaoyan Zhu, Xin Lai +3
Software defect detection is a critical task in software engineering. However, no prior studies have specifically addressed defect detection in bioinformatics software. Given that…
ClaHF: A Human Feedback-inspired Reinforcement Learning Framework for Improving Classification Tasks
Tianxiang Xu, Xiaoyan Zhu, Xin Lai +1
Text classification models are typically trained via supervised fine-tuning (SFT). However, SFT essentially performs behavior cloning from instance-wise labels and thus fails to ad…
Do Papers Tell the Whole Story? A Benchmark and Framework for Uncovering Hidden Implementation Gaps in Bioinformatics
Tianxiang Xu, Xiaoyan Zhu, Xin Lai +4
Ensuring consistency between research papers and their corresponding software code implementations is a fundamental prerequisite for guaranteeing the reproducibility of scientific…
Making Every Head Count: Sparse Attention Without the Speed-Performance Trade-off
Mingkuan Zhao, Wentao Hu, Jiayin Wang +5
The design of Large Language Models (LLMs) has long been hampered by a fundamental conflict within their core attention mechanism: its remarkable expressivity is built upon a compu…
Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Models
Wentao Hu, Mingkuan Zhao, Shuangyong Song +3
Sparse Mixture-of-Experts (SMoE) architectures have enabled a new frontier in scaling Large Language Models (LLMs), offering superior performance by activating only a fraction of t…