8 papers
On the Fragility of Tasks under Composition: Implicit Budgeting in Language Models
Chenpan Li, Thies Brockmoeller, Stefanie Jegelka +2
Modern language models are increasingly asked to handle several tasks within a single prompt. Doing so requires solving each task correctly while integrating the resulting answers…
Canonicalizing Multimodal Contrastive Representation Learning
Sharut Gupta, Sanyam Kansal, Stefanie Jegelka +2
As models and data scale, independently trained networks often induce analogous notions of similarity. But, matching similarities is weaker than establishing an explicit correspond…
Fairness Aware Reward Optimization
Ching Lam Choi, Vighnesh Subramaniam, Phillip Isola +2
Demographic skews in human preference data propagate systematic unfairness through reward models into aligned LLMs. We introduce Fairness Aware Reward Optimization (Faro), an in-pr…
ReasonCACHE: Teaching LLMs To Reason Without Weight Updates
Sharut Gupta, Phillip Isola, Stefanie Jegelka +4
Can Large language models (LLMs) learn to reason without any weight update and only through in-context learning (ICL)? ICL is strikingly sample-efficient, often learning from only…
Learning Diffusion Models with Flexible Representation Guidance
Chenyu Wang, Cai Zhou, Sharut Gupta +4
Diffusion models can be improved with additional guidance towards more effective representations of input. Indeed, prior empirical work has already shown that aligning internal rep…
Sequential-Parallel Duality in Prefix Scannable Models
Morris Yau, Sharut Gupta, Valerie Engelmayer +3
Modern neural sequence models are designed to meet the dual mandate of parallelizable training and fast sequential inference. Recent developments have given rise to various models,…
An Information Criterion for Controlled Disentanglement of Multimodal Data
Chenyu Wang, Sharut Gupta, Xinyi Zhang +4
Multimodal representation learning seeks to relate and decompose information inherent in multiple modalities. By disentangling modality-specific information from information that i…
Tetris-inspired detector with neural network for radiation mapping
Ryotaro Okabe, Shangjie Xue, Jiankai Yu +6
In recent years, radiation mapping has attracted widespread research attention and increased public concerns on environmental monitoring. In terms of both materials and their confi…