12 citations · 32 across the 20 of their papers we have counts for
14 papers · 1 filter
Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
Sijia Liu, Yicheng Lang, Soumyadeep Pal +6
Zeroth-order (ZO) optimization, learning from finite differences of function evaluations without backpropagation, has recently regained attention in deep learning due to its memory…
Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models
Huzaifa Arif, Pin-Yu Chen, Alex Gittens +2
With the increasing reliance on AI models for weather forecasting, it is imperative to evaluate their vulnerability to adversarial perturbations. This work introduces Weather Adapt…
BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models
Evan R. Antoniuk, Shehtab Zaman, Tal Ben-Nun +10
Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules…
Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training
Brian Bartoldson, Siddarth Venkatraman, James Diffenderfer +7
Reinforcement learning (RL) is a critical component of large language model (LLM) post-training. However, on-policy algorithms used for post-training are not naturally robust to a…
End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver
Shaocong Ma, James Diffenderfer, Bhavya Kailkhura +1
Deep learning has been widely applied to solve partial differential equations (PDEs) in computational fluid dynamics. Recent research proposed a PDE correction framework that lever…
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
Jinghan Jia, Yihua Zhang, Yimeng Zhang +5
Large Language Models (LLMs) have highlighted the necessity of effective unlearning mechanisms to comply with data regulations and ethical AI practices. LLM unlearning aims at remo…
Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies
Brian R. Bartoldson, James Diffenderfer, Konstantinos Parasyris +1
This paper revisits the simple, long-studied, yet still unsolved problem of making image classifiers robust to imperceptible perturbations. Taking CIFAR10 as an example, SOTA clean…
DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training
Aochuan Chen, Yimeng Zhang, Jinghan Jia +7
Zeroth-order (ZO) optimization has become a popular technique for solving machine learning (ML) problems when first-order (FO) information is difficult or impossible to obtain. How…
Efficient Multi-Prize Lottery Tickets: Enhanced Accuracy, Training, and Inference Speed
Hao Cheng, Pu Zhao, Yize Li +4
Recently, Diffenderfer and Kailkhura proposed a new paradigm for learning compact yet highly accurate binary neural networks simply by pruning and quantizing randomly weighted full…
Models Out of Line: A Fourier Lens on Distribution Shift Robustness
Sara Fridovich-Keil, Brian R. Bartoldson, James Diffenderfer +2
Improving the accuracy of deep neural networks (DNNs) on out-of-distribution (OOD) data is critical to an acceptance of deep learning (DL) in real world applications. It has been o…
Zeroth-Order SciML: Non-intrusive Integration of Scientific Software with Deep Learning
Ioannis Tsaknakis, Bhavya Kailkhura, Sijia Liu +4
Using deep learning (DL) to accelerate and/or improve scientific workflows can yield discoveries that are otherwise impossible. Unfortunately, DL models have yielded limited succes…
Benchmarking Test-Time Unsupervised Deep Neural Network Adaptation on Edge Devices
Kshitij Bhardwaj, James Diffenderfer, Bhavya Kailkhura +1
The prediction accuracy of the deep neural networks (DNNs) after deployment at the edge can suffer with time due to shifts in the distribution of the new data. To improve robustnes…
A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness
James Diffenderfer, Brian R. Bartoldson, Shreya Chaganti +2
Successful adoption of deep learning (DL) in the wild requires models to be: (1) compact, (2) accurate, and (3) robust to distributional shifts. Unfortunately, efforts towards simu…
Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network
James Diffenderfer, Bhavya Kailkhura
Recently, Frankle & Carbin (2019) demonstrated that randomly-initialized dense networks contain subnetworks that once found can be trained to reach test accuracy comparable to the…