272 citations · 745 across the 34 of their papers we have counts for
34 papers
GPU Sharing with Triples Mode
Chansup Byun, Albert Reuther, LaToya Anderson +19
There is a tremendous amount of interest in AI/ML technologies due to the proliferation of generative AI applications such as ChatGPT. This trend has significantly increased demand…
LLload: An Easy-to-Use HPC Utilization Tool
Chansup Byun, Albert Reuther, Julie Mullen +19
The increasing use and cost of high performance computing (HPC) requires new easy-to-use tools to enable HPC users and HPC systems engineers to transparently understand the utiliza…
HPC with Enhanced User Separation
Andrew Prout, Albert Reuther, Michael Houle +19
HPC systems used for research run a wide variety of software and workflows. This software is often written or modified by users to meet the needs of their research projects, and ra…
pPython Performance Study
Chansup Byun, William Arcand, David Bestor +17
pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics…
pPython for Parallel Python Programming
Chansup Byun, William Arcand, David Bestor +18
pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics…
3D Real-Time Supercomputer Monitoring
Bill Bergeron, Matthew Hubbell, Dylan Sequeira +17
Supercomputers are complex systems producing vast quantities of performance data from multiple sources and of varying types. Performance data from each of the thousands of nodes in…
Node-Based Job Scheduling for Large Scale Simulations of Short Running Jobs
Chansup Byun, William Arcand, David Bestor +15
Diverse workloads such as interactive supercomputing, big data analysis, and large-scale AI algorithm development, requires a high-performance scheduler. This paper presents a nove…
Accuracy and Performance Comparison of Video Action Recognition Approaches
Matthew Hutchinson, Siddharth Samsi, William Arcand +16
Over the past few years, there has been significant interest in video action recognition systems and models. However, direct comparison of accuracy and computational performance re…
Benchmarking network fabrics for data distributed training of deep neural networks
Siddharth Samsi, Andrew Prout, Michael Jones +16
Artificial Intelligence/Machine Learning applications require the training of complex models on large amounts of labelled data. The large computational requirements for training de…
Best of Both Worlds: High Performance Interactive and Batch Launching
Chansup Byun, Jeremy Kepner, William Arcand +16
Rapid launch of thousands of jobs is essential for effective interactive supercomputing, big data analysis, and AI algorithm development. Achieving thousands of launches per second…
Large Scale Parallelization Using File-Based Communications
Chansup Byun, Jeremy Kepner, William Arcand +14
In this paper, we present a novel and new file-based communication architecture using the local filesystem for large scale parallelization. This new approach eliminates the issues…
TapirXLA: Embedding Fork-Join Parallelism into the XLA Compiler in TensorFlow Using Tapir
Tao B. Schardl, Siddharth Samsi
This work introduces TapirXLA, a replacement for TensorFlow's XLA compiler that embeds recursive fork-join parallelism into XLA's low-level representation of code. Machine-learning…
Securing HPC using Federated Authentication
Andrew Prout, William Arcand, David Bestor +15
Federated authentication can drastically reduce the overhead of basic account maintenance while simultaneously improving overall system security. Integrating with the user's more f…
Lessons Learned from a Decade of Providing Interactive, On-Demand High Performance Computing to Scientists and Engineers
Julia Mullen, Albert Reuther, William Arcand +15
For decades, the use of HPC systems was limited to those in the physical sciences who had mastered their domain in conjunction with a deep understanding of HPC architectures and al…
Hyperscaling Internet Graph Analysis with D4M on the MIT SuperCloud
Vijay Gadepally, Jeremy Kepner, Lauren Milechin +14
Detecting anomalous behavior in network traffic is a major challenge due to the volume and velocity of network traffic. For example, a 10 Gigabit Ethernet connection can generate o…
Interactive Launch of 16,000 Microsoft Windows Instances on a Supercomputer
Michael Jones, Jeremy Kepner, Bradley Orchard +16
Simulation, machine learning, and data analysis require a wide range of software which can be dependent upon specific operating systems, such as Microsoft Windows. Running this sof…
Measuring the Impact of Spectre and Meltdown
Andrew Prout, William Arcand, David Bestor +15
The Spectre and Meltdown flaws in modern microprocessors represent a new class of attacks that have been difficult to mitigate. The mitigations that have been proposed have known p…
Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis
Albert Reuther, Jeremy Kepner, Chansup Byun +15
Interactive massively parallel computations are critical for machine learning and data analysis. These computations are a staple of the MIT Lincoln Laboratory Supercomputing Center…
Design, Generation, and Validation of Extreme Scale Power-Law Graphs
Jeremy Kepner, Siddharth Samsi, William Arcand +18
Massive power-law graphs drive many fields: metagenomics, brain mapping, Internet-of-things, cybersecurity, and sparse machine learning. The development of novel algorithms and sys…
Performance Measurements of Supercomputing and Cloud Storage Solutions
Michael Jones, Jeremy Kepner, William Arcand +10
Increasing amounts of data from varied sources, particularly in the fields of machine learning and graph analytics, are causing storage requirements to grow rapidly. A variety of t…
MIT SuperCloud Portal Workspace: Enabling HPC Web Application Deployment
Andrew Prout, William Arcand, David Bestor +13
The MIT SuperCloud Portal Workspace enables the secure exposure of web services running on high performance computing (HPC) systems. The portal allows users to run any web applicat…
Benchmarking Data Analysis and Machine Learning Applications on the Intel KNL Many-Core Processor
Chansup Byun, Jeremy Kepner, William Arcand +15
Knights Landing (KNL) is the code name for the second-generation Intel Xeon Phi product family. KNL has generated significant interest in the data analysis and machine learning com…
Scalable System Scheduling for HPC and Big Data
Albert Reuther, Chansup Byun, William Arcand +8
In the rapidly expanding field of parallel processing, job schedulers are the "operating systems" of modern big data architectures and supercomputing systems. Job schedulers alloca…
Benchmarking SciDB Data Import on HPC Systems
Siddharth Samsi, Laura Brattain, William Arcand +16
SciDB is a scalable, computational database management system that uses an array model for data storage. The array data model of SciDB makes it ideally suited for storing and manag…