10 papers
Circuit Tracing in Autoregressive Protein Language Models
Darin Tsui, William Deinzer, Daniel Saeedi +1
Protein language models (pLMs) can generate novel protein sequences with properties beyond those observed in nature, yet the mechanisms underlying protein generation remain poorly…
On the Error-Correcting Effects of Stochasticity in Discrete Diffusion
William Yuan, Sungwon Jeong, Amirali Aghazadeh
Discrete diffusion models achieve strong performance in text and image generation, but their inference remains slow and must inherently balance sampling efficiency and sample quali…
Structure-Aware Masking for Protein Representation Learning
Thomas Walton, Ayan Goel, Amirali Aghazadeh
Masked language modeling (MLM) is the standard objective for training protein language models, typically implemented by randomly masking individual residues at a fixed rate (e.g.,…
Protein Circuit Tracing via Cross-layer Transcoders
Darin Tsui, Kunal Talreja, Daniel Saeedi +1
Protein language models (pLMs) have emerged as powerful predictors of protein structure and function. However, the computational circuits underlying their predictions remain poorly…
cryoSENSE: Compressive Sensing Enables High-throughput Microscopy with Sparse and Generative Priors on the Protein Cryo-EM Image Manifold
Zain Shabeeb, Daniel Saeedi, Darin Tsui +2
Cryo-electron microscopy (cryo-EM) enables the atomic-resolution visualization of biomolecules; however, modern direct detectors generate data volumes that far exceed the available…
SpecMER: Fast Protein Generation with K-mer Guided Speculative Decoding
Thomas Walton, Darin Tsui, Aryan Musharaf +1
Autoregressive models have transformed protein engineering by enabling the generation of novel protein sequences beyond those found in nature. However, their sequential inference i…