272 citations · 853 across the 57 of their papers we have counts for
32 papers · 1 filter
Easy Acceleration with Distributed Arrays
Jeremy Kepner, Chansup Byun, LaToya Anderson +20
High level programming languages and GPU accelerators are powerful enablers for a wide range of applications. Achieving scalable vertical (within a compute node), horizontal (acros…
GPU Sharing with Triples Mode
Chansup Byun, Albert Reuther, LaToya Anderson +19
There is a tremendous amount of interest in AI/ML technologies due to the proliferation of generative AI applications such as ChatGPT. This trend has significantly increased demand…
Supercomputer 3D Digital Twin for User Focused Real-Time Monitoring
William Bergeron, Matthew Hubbell, Daniel Mojica +17
Real-time supercomputing performance analysis is a critical aspect of evaluating and optimizing computational systems in a dynamic user environment. The operation of supercomputers…
HPC with Enhanced User Separation
Andrew Prout, Albert Reuther, Michael Houle +19
HPC systems used for research run a wide variety of software and workflows. This software is often written or modified by users to meet the needs of their research projects, and ra…
LLload: Simplifying Real-Time Job Monitoring for HPC Users
Chansup Byun, Julia Mullen, Albert Reuther +16
One of the more complex tasks for researchers using HPC systems is performance monitoring and tuning of their applications. Developing a practice of continuous performance improvem…
pPython Performance Study
Chansup Byun, William Arcand, David Bestor +17
pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics…
Python Implementation of the Dynamic Distributed Dimensional Data Model
Hayden Jananthan, Lauren Milechin, Michael Jones +17
Python has become a standard scientific computing language with fast-growing support of machine learning and data analysis modules, as well as an increasing usage of big data. The…
pPython for Parallel Python Programming
Chansup Byun, William Arcand, David Bestor +18
pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics…
The MIT Supercloud Workload Classification Challenge
Benny J. Tang, Qiqi Chen, Matthew L. Weiss +24
High-Performance Computing (HPC) centers and cloud providers support an increasingly diverse set of applications on heterogenous hardware. As Artificial Intelligence (AI) and Machi…
3D Real-Time Supercomputer Monitoring
Bill Bergeron, Matthew Hubbell, Dylan Sequeira +17
Supercomputers are complex systems producing vast quantities of performance data from multiple sources and of varying types. Performance data from each of the thousands of nodes in…
Node-Based Job Scheduling for Large Scale Simulations of Short Running Jobs
Chansup Byun, William Arcand, David Bestor +15
Diverse workloads such as interactive supercomputing, big data analysis, and large-scale AI algorithm development, requires a high-performance scheduler. This paper presents a nove…
Vertical, Temporal, and Horizontal Scaling of Hierarchical Hypersparse GraphBLAS Matrices
Jeremy Kepner, Tim Davis, Chansup Byun +16
Hypersparse matrices are a powerful enabler for a variety of network, health, finance, and social applications. Hierarchical hypersparse GraphBLAS matrices enable rapid streaming u…
The MIT Supercloud Dataset
Siddharth Samsi, Matthew L Weiss, David Bestor +24
Artificial intelligence (AI) and Machine learning (ML) workloads are an increasingly larger share of the compute workloads in traditional High-Performance Computing (HPC) centers a…
Best of Both Worlds: High Performance Interactive and Batch Launching
Chansup Byun, Jeremy Kepner, William Arcand +16
Rapid launch of thousands of jobs is essential for effective interactive supercomputing, big data analysis, and AI algorithm development. Achieving thousands of launches per second…
75,000,000,000 Streaming Inserts/Second Using Hierarchical Hypersparse GraphBLAS Matrices
Jeremy Kepner, Tim Davis, Chansup Byun +16
The SuiteSparse GraphBLAS C-library implements high performance hypersparse matrices with bindings to a variety of languages (Python, Julia, and Matlab/Octave). GraphBLAS provides…
Large Scale Parallelization Using File-Based Communications
Chansup Byun, Jeremy Kepner, William Arcand +14
In this paper, we present a novel and new file-based communication architecture using the local filesystem for large scale parallelization. This new approach eliminates the issues…
Securing HPC using Federated Authentication
Andrew Prout, William Arcand, David Bestor +15
Federated authentication can drastically reduce the overhead of basic account maintenance while simultaneously improving overall system security. Integrating with the user's more f…
Streaming 1.9 Billion Hypersparse Network Updates per Second with D4M
Jeremy Kepner, Vijay Gadepally, Lauren Milechin +15
The Dynamic Distributed Dimensional Data Model (D4M) library implements associative arrays in a variety of languages (Python, Julia, and Matlab/Octave) and provides a lightweight i…
Lessons Learned from a Decade of Providing Interactive, On-Demand High Performance Computing to Scientists and Engineers
Julia Mullen, Albert Reuther, William Arcand +15
For decades, the use of HPC systems was limited to those in the physical sciences who had mastered their domain in conjunction with a deep understanding of HPC architectures and al…
Hyperscaling Internet Graph Analysis with D4M on the MIT SuperCloud
Vijay Gadepally, Jeremy Kepner, Lauren Milechin +14
Detecting anomalous behavior in network traffic is a major challenge due to the volume and velocity of network traffic. For example, a 10 Gigabit Ethernet connection can generate o…
Interactive Launch of 16,000 Microsoft Windows Instances on a Supercomputer
Michael Jones, Jeremy Kepner, Bradley Orchard +16
Simulation, machine learning, and data analysis require a wide range of software which can be dependent upon specific operating systems, such as Microsoft Windows. Running this sof…
Measuring the Impact of Spectre and Meltdown
Andrew Prout, William Arcand, David Bestor +15
The Spectre and Meltdown flaws in modern microprocessors represent a new class of attacks that have been difficult to mitigate. The mitigations that have been proposed have known p…
Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis
Albert Reuther, Jeremy Kepner, Chansup Byun +15
Interactive massively parallel computations are critical for machine learning and data analysis. These computations are a staple of the MIT Lincoln Laboratory Supercomputing Center…
Design, Generation, and Validation of Extreme Scale Power-Law Graphs
Jeremy Kepner, Siddharth Samsi, William Arcand +18
Massive power-law graphs drive many fields: metagenomics, brain mapping, Internet-of-things, cybersecurity, and sparse machine learning. The development of novel algorithms and sys…