From the 1 of 33 linked papers with an AI index.
7 papers · 1 filter
Scalable Cross-Facility Federated Learning for Scientific Foundation Models on Multiple Supercomputers
Yijiang Li, Zilinghan Li, Kyle Chard +4
Artificial Intelligence for scientific applications increasingly requires training large models on data that cannot be centralized due to privacy constraints, data sovereignty, or…
The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators
Mansi Sakarvadia, Kareem Hegazy, Amin Totounferoush +4
A core challenge in scientific machine learning, and scientific computing more generally, is modeling continuous phenomena which (in practice) are represented discretely. Machine-l…
LSHBloom: Memory-efficient, Extreme-scale Document Deduplication
Arham Khan, Robert Underwood, Carlo Siebenschuh +7
Contemporary large language model (LLM) training pipelines require the assembly of internet-scale databases full of text data from a variety of sources (e.g., web, academic, and pu…
Topology-Aware Knowledge Propagation in Decentralized Learning
Mansi Sakarvadia, Nathaniel Hudson, Tian Li +2
Decentralized learning enables collaborative training of models across naturally distributed data without centralized coordination or maintenance of a global model. Instead, device…
Deep Model Merging: The Sister of Neural Network Interpretability -- A Survey
Arham Khan, Todd Nief, Nathaniel Hudson +6
We survey the model merging literature through the lens of loss landscape geometry to connect observations from empirical studies on model merging and loss landscape analysis to ph…
Mitigating Memorization In Language Models
Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6
Language models (LMs) can "memorize" information, i.e., encode training data in their weights in such a way that inference-time queries can lead to verbatim regurgitation of that d…