272 citations · 1.5k across the 62 of their papers we have counts for
34 papers · 1 filter
LLload: Simplifying Real-Time Job Monitoring for HPC Users
Chansup Byun, Julia Mullen, Albert Reuther +16
One of the more complex tasks for researchers using HPC systems is performance monitoring and tuning of their applications. Developing a practice of continuous performance improvem…
pPython Performance Study
Chansup Byun, William Arcand, David Bestor +17
pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics…
Python Implementation of the Dynamic Distributed Dimensional Data Model
Hayden Jananthan, Lauren Milechin, Michael Jones +17
Python has become a standard scientific computing language with fast-growing support of machine learning and data analysis modules, as well as an increasing usage of big data. The…
pPython for Parallel Python Programming
Chansup Byun, William Arcand, David Bestor +18
pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics…
The MIT Supercloud Workload Classification Challenge
Benny J. Tang, Qiqi Chen, Matthew L. Weiss +24
High-Performance Computing (HPC) centers and cloud providers support an increasingly diverse set of applications on heterogenous hardware. As Artificial Intelligence (AI) and Machi…
3D Real-Time Supercomputer Monitoring
Bill Bergeron, Matthew Hubbell, Dylan Sequeira +17
Supercomputers are complex systems producing vast quantities of performance data from multiple sources and of varying types. Performance data from each of the thousands of nodes in…
Node-Based Job Scheduling for Large Scale Simulations of Short Running Jobs
Chansup Byun, William Arcand, David Bestor +15
Diverse workloads such as interactive supercomputing, big data analysis, and large-scale AI algorithm development, requires a high-performance scheduler. This paper presents a nove…
Vertical, Temporal, and Horizontal Scaling of Hierarchical Hypersparse GraphBLAS Matrices
Jeremy Kepner, Tim Davis, Chansup Byun +16
Hypersparse matrices are a powerful enabler for a variety of network, health, finance, and social applications. Hierarchical hypersparse GraphBLAS matrices enable rapid streaming u…
The MIT Supercloud Dataset
Siddharth Samsi, Matthew L Weiss, David Bestor +24
Artificial intelligence (AI) and Machine learning (ML) workloads are an increasingly larger share of the compute workloads in traditional High-Performance Computing (HPC) centers a…
Survey of Machine Learning Accelerators
Albert Reuther, Peter Michaleas, Michael Jones +3
New machine learning accelerators are being announced and released each month for a variety of applications from speech recognition, video object detection, assisted driving, and m…
Compute, Time and Energy Characterization of Encoder-Decoder Networks with Automatic Mixed Precision Training
Siddharth Samsi, Michael Jones, Mark M. Veillette
Deep neural networks have shown great success in many diverse fields. The training of these networks can take significant amounts of time, compute and energy. As datasets get large…
Benchmarking network fabrics for data distributed training of deep neural networks
Siddharth Samsi, Andrew Prout, Michael Jones +16
Artificial Intelligence/Machine Learning applications require the training of complex models on large amounts of labelled data. The large computational requirements for training de…
Best of Both Worlds: High Performance Interactive and Batch Launching
Chansup Byun, Jeremy Kepner, William Arcand +16
Rapid launch of thousands of jobs is essential for effective interactive supercomputing, big data analysis, and AI algorithm development. Achieving thousands of launches per second…
Fast Mapping onto Census Blocks
Jeremy Kepner, Andreas Kipf, Darren Engwirda +21
Pandemic measures such as social distancing and contact tracing can be enhanced by rapidly integrating dynamic location data and demographic data. Projecting billions of longitude…
GraphChallenge.org Triangle Counting Performance
Siddharth Samsi, Jeremy Kepner, Vijay Gadepally +9
The rise of graph analytic systems has created a need for new ways to measure and compare the capabilities of graph processing systems. The MIT/Amazon/IEEE Graph Challenge has been…
75,000,000,000 Streaming Inserts/Second Using Hierarchical Hypersparse GraphBLAS Matrices
Jeremy Kepner, Tim Davis, Chansup Byun +16
The SuiteSparse GraphBLAS C-library implements high performance hypersparse matrices with bindings to a variety of languages (Python, Julia, and Matlab/Octave). GraphBLAS provides…
Large Scale Parallelization Using File-Based Communications
Chansup Byun, Jeremy Kepner, William Arcand +14
In this paper, we present a novel and new file-based communication architecture using the local filesystem for large scale parallelization. This new approach eliminates the issues…
Securing HPC using Federated Authentication
Andrew Prout, William Arcand, David Bestor +15
Federated authentication can drastically reduce the overhead of basic account maintenance while simultaneously improving overall system security. Integrating with the user's more f…
Streaming 1.9 Billion Hypersparse Network Updates per Second with D4M
Jeremy Kepner, Vijay Gadepally, Lauren Milechin +15
The Dynamic Distributed Dimensional Data Model (D4M) library implements associative arrays in a variety of languages (Python, Julia, and Matlab/Octave) and provides a lightweight i…
Lessons Learned from a Decade of Providing Interactive, On-Demand High Performance Computing to Scientists and Engineers
Julia Mullen, Albert Reuther, William Arcand +15
For decades, the use of HPC systems was limited to those in the physical sciences who had mastered their domain in conjunction with a deep understanding of HPC architectures and al…
Hyperscaling Internet Graph Analysis with D4M on the MIT SuperCloud
Vijay Gadepally, Jeremy Kepner, Lauren Milechin +14
Detecting anomalous behavior in network traffic is a major challenge due to the volume and velocity of network traffic. For example, a 10 Gigabit Ethernet connection can generate o…
Interactive Launch of 16,000 Microsoft Windows Instances on a Supercomputer
Michael Jones, Jeremy Kepner, Bradley Orchard +16
Simulation, machine learning, and data analysis require a wide range of software which can be dependent upon specific operating systems, such as Microsoft Windows. Running this sof…
Measuring the Impact of Spectre and Meltdown
Andrew Prout, William Arcand, David Bestor +15
The Spectre and Meltdown flaws in modern microprocessors represent a new class of attacks that have been difficult to mitigate. The mitigations that have been proposed have known p…
Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis
Albert Reuther, Jeremy Kepner, Chansup Byun +15
Interactive massively parallel computations are critical for machine learning and data analysis. These computations are a staple of the MIT Lincoln Laboratory Supercomputing Center…