82 citations · 231 across the 49 of their papers we have counts for
11 papers · 1 filter
Optimizing Distributed Training on Frontier for Large Language Models
Sajal Dash, Isaac Lyngaas, Junqi Yin +5
Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Recent studies on loss sca…
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang +89
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scient…
Self-supervised Learning for Anomaly Detection in Computational Workflows
Hongwei Jin, Krishnan Raghavan, George Papadimitriou +4
Anomaly detection is the task of identifying abnormal behavior of a system. Anomaly detection in computational workflows is of special interest because of its wide implications in…
Parallel Multi-Objective Hyperparameter Optimization with Uniform Normalization and Bounded Objectives
Romain Egele, Tyler Chang, Yixuan Sun +2
Machine learning (ML) methods offer a wide range of configurable hyperparameters that have a significant influence on their performance. While accuracy is a commonly used performan…
Improving Performance in Continual Learning Tasks using Bio-Inspired Architectures
Sandeep Madireddy, Angel Yanguas-Gil, Prasanna Balaprakash
The ability to learn continuously from an incoming data stream without catastrophic forgetting is critical to designing intelligent systems. Many approaches to continual learning r…
Is One Epoch All You Need For Multi-Fidelity Hyperparameter Optimization?
Romain Egele, Isabelle Guyon, Yixuan Sun +1
Hyperparameter optimization (HPO) is crucial for fine-tuning machine learning models but can be computationally expensive. To reduce costs, Multi-fidelity HPO (MF-HPO) leverages in…
Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search
Shengli Jiang, Shiyi Qin, Reid C. Van Lehn +2
Graph Neural Networks (GNNs) have emerged as a prominent class of data-driven methods for molecular property prediction. However, a key limitation of typical GNN models is their in…
Learning Continually on a Sequence of Graphs -- The Dynamical System Way
Krishnan Raghavan, Prasanna Balaprakash
Continual learning~(CL) is a field concerned with learning a series of inter-related task with the tasks typically defined in the sense of either regression or classification. In r…
Application of probabilistic modeling and automated machine learning framework for high-dimensional stress field
Lele Luan, Nesar Ramachandra, Sandipp Krishnan Ravi +6
Modern computational methods, involving highly sophisticated mathematical formulations, enable several tasks like modeling complex physical phenomenon, predicting key properties an…
Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles
Romit Maulik, Romain Egele, Krishnan Raghavan +1
Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DN…
Analyzing the impact of climate change on critical infrastructure from the scientific literature: A weakly supervised NLP approach
Tanwi Mallick, Joshua David Bergerson, Duane R. Verner +3
Natural language processing (NLP) is a promising approach for analyzing large volumes of climate-change and infrastructure-related scientific literature. However, best-in-practice…