68 citations · 164 across the 8 of their papers we have counts for
8 papers
Revisiting BFloat16 Training
Pedram Zamirai, Jian Zhang, Christopher R. Aberger +1
State-of-the-art generic low-precision training algorithms use a mix of 16-bit and 32-bit precision, creating the folklore that 16-bit hardware compute units alone are not enough t…
Understanding the Downstream Instability of Word Embeddings
Megan Leszczynski, Avner May, Jian Zhang +3
Many industrial machine learning (ML) systems require frequent retraining to keep up-to-date with constantly changing data. This retraining exacerbates a large challenge facing ML…
PipeMare: Asynchronous Pipeline Parallel DNN Training
Bowen Yang, Jian Zhang, Jonathan Li +3
Pipeline parallelism (PP) when training neural networks enables larger models to be partitioned spatially, leading to both lower network communication and overall higher hardware u…
Low-Memory Neural Network Training: A Technical Report
Nimit S. Sohoni, Christopher R. Aberger, Megan Leszczynski +2
Memory is increasingly often the bottleneck when training neural network models. Despite this, techniques to lower the overall memory requirements of training have been less widely…
High-Accuracy Low-Precision Training
Christopher De Sa, Megan Leszczynski, Jian Zhang +4
Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has b…
LevelHeaded: Making Worst-Case Optimal Joins Work in the Common Case
Christopher R. Aberger, Andrew Lamb, Kunle Olukotun +1
Pipelines combining SQL-style business intelligence (BI) queries and linear algebra (LA) are becoming increasingly common in industry. As a result, there is a growing need to unify…
Old Techniques for New Join Algorithms: A Case Study in RDF Processing
Christopher R. Aberger, Susan Tu, Kunle Olukotun +1
Recently there has been significant interest around designing specialized RDF engines, as traditional query processing mechanisms incur orders of magnitude performance gaps on many…
EmptyHeaded: A Relational Engine for Graph Processing
Christopher R. Aberger, Susan Tu, Kunle Olukotun +1
There are two types of high-performance graph processing engines: low- and high-level engines. Low-level engines (Galois, PowerGraph, Snap) provide optimized data structures and co…