8 papers
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Xingyu Su, Jacob Helwig, Shubham Parashar +6
We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs). Rather than pretraining from scratch, prior work replaces the causal attention i…
Learnability-Informed Fine-Tuning of Diffusion Language Models
Shubham Parashar, Atharv Chagi, Jacob Helwig +5
We aim to improve the reasoning capabilities of diffusion language models (DLMs). While SFT is a popular post-training recipe for autoregressive models, its use in DLMs faces chall…
Orbital Transformers for Predicting Wavefunctions in Time-Dependent Density Functional Theory
Xuan Zhang, Haiyang Yu, Chengdong Wang +3
We aim to learn wavefunctions simulated by time-dependent density functional theory (TDDFT), which can be efficiently represented as linear combination coefficients of atomic orbit…
A Two-Phase Deep Learning Framework for Adaptive Time-Stepping in High-Speed Flow Modeling
Jacob Helwig, Sai Sreeharsha Adavi, Xuan Zhang +11
We consider the problem of modeling high-speed flows using machine learning methods. While most prior studies focus on low-speed fluid flows in which uniform time-stepping is pract…
Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency
Yuchao Lin, Jacob Helwig, Shurui Gui +1
We consider achieving equivariance in machine learning systems via frame averaging. Current frame averaging methods involve a costly sum over large frames or rely on sampling-based…
Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
Xuan Zhang, Limei Wang, Jacob Helwig +60
Advances in artificial intelligence (AI) are fueling a new paradigm of discoveries in natural sciences. Today, AI has started to advance natural sciences by improving, accelerating…