7 papers
Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal
Leon Bergen, Usha Bhalla, Sidharth Baskaran +14
Language-model post-training is the main stage at which model behavior is shaped, yet it still largely involves optimization of scalar rewards that summarize diverse desiderata. Th…
CT Open: An Open-Access, Uncontaminated, Live Platform for the Open Challenge of Clinical Trial Outcome Prediction
Jianyou Wang, Youze Zheng, Longtian Bao +11
Scientists have long sought to accurately predict outcomes of real-world events before they happen. Can AI systems do so more reliably? We study this question through clinical tria…
Quiet Feature Learning in Algorithmic Tasks
Prudhviraj Naidu, Zixian Wang, Leon Bergen +1
We train Transformer-based language models on ten foundational algorithmic tasks and observe pronounced phase transitions in their loss curves that deviate from established power-l…
Measuring Risk of Bias in Biomedical Reports: The RoBBR Benchmark
Jianyou Wang, Weili Cao, Longtian Bao +6
Systems that answer questions by reviewing the scientific literature are becoming increasingly feasible. To draw reliable conclusions, these systems should take into account the qu…
EvidenceBench: A Benchmark for Extracting Evidence from Biomedical Papers
Jianyou Wang, Weili Cao, Kaicheng Wang +13
We study the task of automatically finding evidence relevant to hypotheses in biomedical papers. Finding relevant evidence is an important step when researchers investigate scienti…
Single-Pass Document Scanning for Question Answering
Weili Cao, Jianyou Wang, Youze Zheng +5
Handling extremely large documents for question answering is challenging: chunk-based embedding methods often lose track of important global context, while full-context transformer…