3 citations · 8 across the 12 of their papers we have counts for
14 papers
Initialization Improves LLM-Driven Discovery
Mansi Sakarvadia, Marco Ciccone, Colin Raffel
Large Language Models (LLMs) have been used for novel discovery of algorithms, theorems, drugs, and other tasks through the use of harnesses that prompt an LLM to iteratively optim…
Extremes on Rewind: Generating 1,000-Member Ensembles Initialized at a Final Condition
Jerry Lin, Mu-Ting Chien, Mansi Sakarvadia +1
Scenario planning for rare, high-impact events often requires massive ensembles to stochastically sample relevant trajectories. Although autoregressive weather emulators can effici…
Interpreting Language Model Hidden States at Scale
Jordan Pettyjohn, Mansi Sakarvadia, Nathaniel Hudson +3
Lens methods interpret large language models (LLMs) by mapping intermediate activations to the output vocabulary, revealing how next-token predictions develop through the network.…
The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators
Mansi Sakarvadia, Kareem Hegazy, Amin Totounferoush +4
A core challenge in scientific machine learning, and scientific computing more generally, is modeling continuous phenomena which (in practice) are represented discretely. Machine-l…
Topology-Aware Knowledge Propagation in Decentralized Learning
Mansi Sakarvadia, Nathaniel Hudson, Tian Li +2
Decentralized learning enables collaborative training of models across naturally distributed data without centralized coordination or maintenance of a global model. Instead, device…
Empowering Scientific Workflows with Federated Agents
Alok Kamatar, J. Gregory Pauloski, Yadu Babuji +5
Agentic systems, in which diverse agents cooperate to tackle challenging problems, are exploding in popularity in the AI community. However, existing agentic frameworks take a rela…
SCOPE-MRI: Bankart Lesion Detection as a Case Study in Data Curation and Deep Learning for Challenging Diagnoses
Sahil Sethi, Sai Reddy, Mansi Sakarvadia +4
Deep learning has shown strong performance in musculoskeletal imaging, but prior work has largely targeted conditions where diagnosis is relatively straightforward. More challengin…
Toward Non-Invasive Diagnosis of Bankart Lesions with Deep Learning
Sahil Sethi, Sai Reddy, Mansi Sakarvadia +4
Bankart lesions, or anterior-inferior glenoid labral tears, are diagnostically challenging on standard MRIs due to their subtle imaging features-often necessitating invasive MRI ar…
Towards Interpreting Language Models: A Case Study in Multi-Hop Reasoning
Mansi Sakarvadia
Answering multi-hop reasoning questions requires retrieving and synthesizing information from diverse sources. Language models (LMs) struggle to perform such reasoning consistently…
Deep Model Merging: The Sister of Neural Network Interpretability -- A Survey
Arham Khan, Todd Nief, Nathaniel Hudson +6
We survey the model merging literature through the lens of loss landscape geometry to connect observations from empirical studies on model merging and loss landscape analysis to ph…
Mitigating Memorization In Language Models
Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6
Language models (LMs) can "memorize" information, i.e., encode training data in their weights in such a way that inference-time queries can lead to verbatim regurgitation of that d…
Trillion Parameter AI Serving Infrastructure for Scientific Discovery: A Survey and Vision
Nathaniel Hudson, J. Gregory Pauloski, Matt Baughman +13
Deep learning methods are transforming research, enabling new techniques, and ultimately leading to new discoveries. As the demand for more capable AI models continues to grow, we…
Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanism
Mansi Sakarvadia, Arham Khan, Aswathy Ajith +5
Transformer-based Large Language Models (LLMs) are the state-of-the-art for natural language tasks. Recent work has attempted to decode, by reverse engineering the role of linear l…
Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models
Mansi Sakarvadia, Aswathy Ajith, Arham Khan +5
Answering multi-hop reasoning questions requires retrieving and synthesizing information from diverse sources. Large Language Models (LLMs) struggle to perform such reasoning consi…