4 papers
BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data
Brian Hsu, Ozan Gökdemir, Carlo Siebenschuh +7
Despite the large corpus of biology training text, the impact of reasoning models on biological research generally lags behind math and coding. In this work, we show that biology q…
VILLA: Versatile Information Retrieval From Scientific Literature Using Large LAnguage Models
Blessy Antony, Amartya Dutta, Sneha Aggarwal +7
The lack of high-quality ground truth datasets to train machine learning (ML) models impedes the potential of artificial intelligence (AI) for science research. Scientific informat…
Self Distillation Fine-Tuning of Protein Language Models Improves Versatility in Protein Design
Amin Tavakoli, Raswanth Murugan, Ozan Gokdemir +3
Supervised fine-tuning (SFT) is a standard approach for adapting large language models to specialized domains, yet its application to protein sequence modeling and protein language…
LSHBloom: Memory-efficient, Extreme-scale Document Deduplication
Arham Khan, Robert Underwood, Carlo Siebenschuh +7
Contemporary large language model (LLM) training pipelines require the assembly of internet-scale databases full of text data from a variety of sources (e.g., web, academic, and pu…