82 citations · 231 across the 49 of their papers we have counts for
27 papers · 1 filter
Large Language Models Inference Engines based on Spiking Neural Networks
Adarsha Balaji, Sandeep Madireddy, Prasanna Balaprakash
Foundational models based on the transformer architecture are currently the state-of-the-art in general language modeling, as well as in scientific areas such as material science a…
Multi-task parallelism for robust pre-training of graph foundation models on multi-source, multi-fidelity atomistic modeling data
Massimiliano Lupo Pasini, Jong Youl Choi, Pei Zhang +6
Graph foundation models using graph neural networks promise sustainable, efficient atomistic modeling. To tackle challenges of processing multi-source, multi-fidelity data during p…
Distributed Cross-Channel Hierarchical Aggregation for Foundation Models
Aristeidis Tsaris, Isaac Lyngaas, John Lagregren +6
Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images fr…
ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling
Xiao Wang, Jong-Youl Choi, Takuya Kurihaya +15
Sparse observations and coarse-resolution climate models limit effective regional decision-making, underscoring the need for robust downscaling. However, existing AI methods strugg…
Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling
Chaojian Li, Zhifan Ye, Massimiliano Lupo Pasini +4
Atomistic materials modeling is a critical task with wide-ranging applications, from drug discovery to materials science, where accurate predictions of the target material property…
Bayesian optimized deep ensemble for uncertainty quantification of deep neural networks: a system safety case study on sodium fast reactor thermal stratification modeling
Zaid Abulawi, Rui Hu, Prasanna Balaprakash +1
Accurate predictions and uncertainty quantification (UQ) are essential for decision-making in risk-sensitive fields such as system safety modeling. Deep ensembles (DEs) are efficie…
Generalizable Prediction Model of Molten Salt Mixture Density with Chemistry-Informed Transfer Learning
Julian Barra, Shayan Shahbazi, Anthony Birri +6
Optimally designing molten salt applications requires knowledge of their thermophysical properties, but existing databases are incomplete, and experiments are challenging. Ideal mi…
Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNN
Massimiliano Lupo Pasini, Jong Youl Choi, Kshitij Mehta +8
We present our work on developing and training scalable, trustworthy, and energy-efficient predictive graph foundation models (GFMs) using HydraGNN, a multi-headed graph convolutio…
Network architecture search of X-ray based scientific applications
Adarsha Balaji, Ramyad Hadidi, Gregory Kollmer +2
X-ray and electron diffraction-based microscopy use bragg peak detection and ptychography to perform 3-D imaging at an atomic resolution. Typically, these techniques are implemente…
AI Competitions and Benchmarks: Dataset Development
Romain Egele, Julio C. S. Jacques Junior, Jan N. van Rijn +7
Machine learning is now used in many applications thanks to its ability to predict, generate, or discover patterns from large quantities of data. However, the process of collecting…
Streamlining Ocean Dynamics Modeling with Fourier Neural Operators: A Multiobjective Hyperparameter and Architecture Optimization Approach
Yixuan Sun, Ololade Sowunmi, Romain Egele +3
Training an effective deep learning model to learn ocean processes involves careful choices of various hyperparameters. We leverage the advanced search algorithms for multiobjectiv…
The Unreasonable Effectiveness Of Early Discarding After One Epoch In Neural Network Hyperparameter Optimization
Romain Egele, Felix Mohr, Tom Viering +1
To reach high performance with deep learning, hyperparameter optimization (HPO) is essential. This process is usually time-consuming due to costly evaluations of neural networks. E…
Self-supervised Learning for Anomaly Detection in Computational Workflows
Hongwei Jin, Krishnan Raghavan, George Papadimitriou +4
Anomaly detection is the task of identifying abnormal behavior of a system. Anomaly detection in computational workflows is of special interest because of its wide implications in…
Parallel Multi-Objective Hyperparameter Optimization with Uniform Normalization and Bounded Objectives
Romain Egele, Tyler Chang, Yixuan Sun +2
Machine learning (ML) methods offer a wide range of configurable hyperparameters that have a significant influence on their performance. While accuracy is a commonly used performan…
Improving Performance in Continual Learning Tasks using Bio-Inspired Architectures
Sandeep Madireddy, Angel Yanguas-Gil, Prasanna Balaprakash
The ability to learn continuously from an incoming data stream without catastrophic forgetting is critical to designing intelligent systems. Many approaches to continual learning r…
Is One Epoch All You Need For Multi-Fidelity Hyperparameter Optimization?
Romain Egele, Isabelle Guyon, Yixuan Sun +1
Hyperparameter optimization (HPO) is crucial for fine-tuning machine learning models but can be computationally expensive. To reduce costs, Multi-fidelity HPO (MF-HPO) leverages in…
Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search
Shengli Jiang, Shiyi Qin, Reid C. Van Lehn +2
Graph Neural Networks (GNNs) have emerged as a prominent class of data-driven methods for molecular property prediction. However, a key limitation of typical GNN models is their in…
Learning Continually on a Sequence of Graphs -- The Dynamical System Way
Krishnan Raghavan, Prasanna Balaprakash
Continual learning~(CL) is a field concerned with learning a series of inter-related task with the tasks typically defined in the sense of either regression or classification. In r…
Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles
Romit Maulik, Romain Egele, Krishnan Raghavan +1
Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DN…
Analyzing the impact of climate change on critical infrastructure from the scientific literature: A weakly supervised NLP approach
Tanwi Mallick, Joshua David Bergerson, Duane R. Verner +3
Natural language processing (NLP) is a promising approach for analyzing large volumes of climate-change and infrastructure-related scientific literature. However, best-in-practice…
Unified Probabilistic Neural Architecture and Weight Ensembling Improves Model Robustness
Sumegha Premchandar, Sandeep Madireddy, Sanket Jantre +1
Robust machine learning models with accurately calibrated uncertainties are crucial for safety-critical applications. Probabilistic machine learning and especially the Bayesian for…
Explainable Graph Pyramid Autoformer for Long-Term Traffic Forecasting
Weiheng Zhong, Tanwi Mallick, Hadi Meidani +2
Accurate traffic forecasting is vital to an intelligent transportation system. Although many deep learning models have achieved state-of-art performance for short-term traffic fore…
Asynchronous Decentralized Bayesian Optimization for Large Scale Hyperparameter Optimization
Romain Egele, Isabelle Guyon, Venkatram Vishwanath +1
Bayesian optimization (BO) is a promising approach for hyperparameter optimization of deep neural networks (DNNs), where each model training can take minutes to hours. In BO, a com…
Multifidelity Reinforcement Learning with Control Variates
Sami Khairy, Prasanna Balaprakash
In many computational science and engineering applications, the output of a system of interest corresponding to a given input can be queried at different levels of fidelity with di…