272 citations · 716 across the 54 of their papers we have counts for
29 papers · 1 filter
Easy Acceleration with Distributed Arrays
Jeremy Kepner, Chansup Byun, LaToya Anderson +20
High level programming languages and GPU accelerators are powerful enablers for a wide range of applications. Achieving scalable vertical (within a compute node), horizontal (acros…
GPU Sharing with Triples Mode
Chansup Byun, Albert Reuther, LaToya Anderson +19
There is a tremendous amount of interest in AI/ML technologies due to the proliferation of generative AI applications such as ChatGPT. This trend has significantly increased demand…
HPC with Enhanced User Separation
Andrew Prout, Albert Reuther, Michael Houle +19
HPC systems used for research run a wide variety of software and workflows. This software is often written or modified by users to meet the needs of their research projects, and ra…
LLload: Simplifying Real-Time Job Monitoring for HPC Users
Chansup Byun, Julia Mullen, Albert Reuther +16
One of the more complex tasks for researchers using HPC systems is performance monitoring and tuning of their applications. Developing a practice of continuous performance improvem…
pPython Performance Study
Chansup Byun, William Arcand, David Bestor +17
pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics…
Python Implementation of the Dynamic Distributed Dimensional Data Model
Hayden Jananthan, Lauren Milechin, Michael Jones +17
Python has become a standard scientific computing language with fast-growing support of machine learning and data analysis modules, as well as an increasing usage of big data. The…
pPython for Parallel Python Programming
Chansup Byun, William Arcand, David Bestor +18
pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics…
The MIT Supercloud Workload Classification Challenge
Benny J. Tang, Qiqi Chen, Matthew L. Weiss +24
High-Performance Computing (HPC) centers and cloud providers support an increasingly diverse set of applications on heterogenous hardware. As Artificial Intelligence (AI) and Machi…
3D Real-Time Supercomputer Monitoring
Bill Bergeron, Matthew Hubbell, Dylan Sequeira +17
Supercomputers are complex systems producing vast quantities of performance data from multiple sources and of varying types. Performance data from each of the thousands of nodes in…
Node-Based Job Scheduling for Large Scale Simulations of Short Running Jobs
Chansup Byun, William Arcand, David Bestor +15
Diverse workloads such as interactive supercomputing, big data analysis, and large-scale AI algorithm development, requires a high-performance scheduler. This paper presents a nove…
Vertical, Temporal, and Horizontal Scaling of Hierarchical Hypersparse GraphBLAS Matrices
Jeremy Kepner, Tim Davis, Chansup Byun +16
Hypersparse matrices are a powerful enabler for a variety of network, health, finance, and social applications. Hierarchical hypersparse GraphBLAS matrices enable rapid streaming u…
Benchmarking network fabrics for data distributed training of deep neural networks
Siddharth Samsi, Andrew Prout, Michael Jones +16
Artificial Intelligence/Machine Learning applications require the training of complex models on large amounts of labelled data. The large computational requirements for training de…
Best of Both Worlds: High Performance Interactive and Batch Launching
Chansup Byun, Jeremy Kepner, William Arcand +16
Rapid launch of thousands of jobs is essential for effective interactive supercomputing, big data analysis, and AI algorithm development. Achieving thousands of launches per second…
75,000,000,000 Streaming Inserts/Second Using Hierarchical Hypersparse GraphBLAS Matrices
Jeremy Kepner, Tim Davis, Chansup Byun +16
The SuiteSparse GraphBLAS C-library implements high performance hypersparse matrices with bindings to a variety of languages (Python, Julia, and Matlab/Octave). GraphBLAS provides…
Large Scale Parallelization Using File-Based Communications
Chansup Byun, Jeremy Kepner, William Arcand +14
In this paper, we present a novel and new file-based communication architecture using the local filesystem for large scale parallelization. This new approach eliminates the issues…
Securing HPC using Federated Authentication
Andrew Prout, William Arcand, David Bestor +15
Federated authentication can drastically reduce the overhead of basic account maintenance while simultaneously improving overall system security. Integrating with the user's more f…
Streaming 1.9 Billion Hypersparse Network Updates per Second with D4M
Jeremy Kepner, Vijay Gadepally, Lauren Milechin +15
The Dynamic Distributed Dimensional Data Model (D4M) library implements associative arrays in a variety of languages (Python, Julia, and Matlab/Octave) and provides a lightweight i…
Lessons Learned from a Decade of Providing Interactive, On-Demand High Performance Computing to Scientists and Engineers
Julia Mullen, Albert Reuther, William Arcand +15
For decades, the use of HPC systems was limited to those in the physical sciences who had mastered their domain in conjunction with a deep understanding of HPC architectures and al…
Hyperscaling Internet Graph Analysis with D4M on the MIT SuperCloud
Vijay Gadepally, Jeremy Kepner, Lauren Milechin +14
Detecting anomalous behavior in network traffic is a major challenge due to the volume and velocity of network traffic. For example, a 10 Gigabit Ethernet connection can generate o…
Interactive Launch of 16,000 Microsoft Windows Instances on a Supercomputer
Michael Jones, Jeremy Kepner, Bradley Orchard +16
Simulation, machine learning, and data analysis require a wide range of software which can be dependent upon specific operating systems, such as Microsoft Windows. Running this sof…
Measuring the Impact of Spectre and Meltdown
Andrew Prout, William Arcand, David Bestor +15
The Spectre and Meltdown flaws in modern microprocessors represent a new class of attacks that have been difficult to mitigate. The mitigations that have been proposed have known p…
Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis
Albert Reuther, Jeremy Kepner, Chansup Byun +15
Interactive massively parallel computations are critical for machine learning and data analysis. These computations are a staple of the MIT Lincoln Laboratory Supercomputing Center…
Design, Generation, and Validation of Extreme Scale Power-Law Graphs
Jeremy Kepner, Siddharth Samsi, William Arcand +18
Massive power-law graphs drive many fields: metagenomics, brain mapping, Internet-of-things, cybersecurity, and sparse machine learning. The development of novel algorithms and sys…
MIT SuperCloud Portal Workspace: Enabling HPC Web Application Deployment
Andrew Prout, William Arcand, David Bestor +13
The MIT SuperCloud Portal Workspace enables the secure exposure of web services running on high performance computing (HPC) systems. The portal allows users to run any web applicat…