4 papers
BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data
Brian Hsu, Ozan Gökdemir, Carlo Siebenschuh +7
Despite the large corpus of biology training text, the impact of reasoning models on biological research generally lags behind math and coding. In this work, we show that biology q…
LSHBloom: Memory-efficient, Extreme-scale Document Deduplication
Arham Khan, Robert Underwood, Carlo Siebenschuh +7
Contemporary large language model (LLM) training pipelines require the assembly of internet-scale databases full of text data from a variety of sources (e.g., web, academic, and pu…
Self Distillation Fine-Tuning of Protein Language Models Improves Versatility in Protein Design
Amin Tavakoli, Raswanth Murugan, Ozan Gokdemir +3
Supervised fine-tuning (SFT) is a standard approach for adapting large language models to specialized domains, yet its application to protein sequence modeling and protein language…
AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling Engine
Carlo Siebenschuh, Kyle Hippe, Ozan Gokdemir +10
Language models for scientific tasks are trained on text from scientific publications, most distributed as PDFs that require parsing. PDF parsing approaches range from inexpensive…