2 citations · 4 across the 12 of their papers we have counts for
13 papers
Unstructured Hydrodynamics on Spatial Dataflow Architectures: A Joint Code and Data Decomposition Approach
Piotr Luczynski, Tal Ben-Nun, Leighton Wilson +1
Spatial Dataflow Architectures are an emerging hardware pattern in high-performance computing, whose mesh-connected fixed-memory processing elements are tailored for structured gri…
LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning
Sumeet Ramesh Motwani, Daniel Nichols, Charles London +17
As language models are increasingly deployed for complex autonomous tasks, their ability to reason accurately over longer horizons becomes critical. An essential component of this…
Record-Remix-Replay: Hierarchical GPU Kernel Optimization using Evolutionary Search
Daniel Nichols, Konstantinos Parasyris, Caetano Melone +3
As high-performance computing and AI workloads become increasingly dependent on GPUs, maintaining high performance across rapidly evolving hardware generations has become a major c…
Inductive Loop Analysis for Practical HPC Application Optimization
Philipp Schaad, Tal Ben-Nun, Patrick Iff +1
Scientific computing applications heavily rely on multi-level loop nests operating on multidimensional arrays. This presents multiple optimization opportunities from exploiting par…
VEIL: Reading Control Flow Graphs Like Code
Philipp Schaad, Tal Ben-Nun, Torsten Hoefler
Control flow graphs (CFGs) are essential tools for understanding program behavior, yet the size of real-world CFGs makes them difficult to interpret. With thousands of nodes and ed…
SpaDA: A Spatial Dataflow Architecture Programming Language
Lukas Gianinazzi, Tal Ben-Nun, Torsten Hoefler
Spatial dataflow architectures like the Cerebras Wafer-Scale Engine deliver exceptional performance in AI and scientific computing by distributing scratchpad memory across hundreds…
Modeling Code: Is Text All You Need?
Daniel Nichols, Konstantinos Parasyris, Harshitha Menon +4
Code LLMs have become extremely popular recently for modeling source code across a variety of tasks, such as generation, translation, and summarization. However, transformer-based…
Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directions
Keita Teranishi, Harshitha Menon, William F. Godoy +25
We discuss the challenges and propose research directions for using AI to revolutionize the development of high-performance computing (HPC) software. AI technologies, in particular…
A Unifying Framework to Enable Artificial Intelligence in High Performance Computing Workflows
Jens Domke, Mohamed Wahib, Anshu Dubey +2
Current trends point to a future where large-scale scientific applications are tightly-coupled HPC/AI hybrids. Hence, we urgently need to invest in creating a seamless, scalable fr…
BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models
Evan R. Antoniuk, Shehtab Zaman, Tal Ben-Nun +10
Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules…
Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training
Brian Bartoldson, Siddarth Venkatraman, James Diffenderfer +7
Reinforcement learning (RL) is a critical component of large language model (LLM) post-training. However, on-policy algorithms used for post-training are not naturally robust to a…
Lion Cub: Minimizing Communication Overhead in Distributed Lion
Satoki Ishikawa, Tal Ben-Nun, Brian Van Essen +2
Communication overhead is a key challenge in distributed deep learning, especially on slower Ethernet interconnects, and given current hardware trends, communication is likely to b…
Low-Depth Spatial Tree Algorithms
Yves Baumann, Tal Ben-Nun, Maciej Besta +3
Contemporary accelerator designs exhibit a high degree of spatial localization, wherein two-dimensional physical distance determines communication costs between processing elements…