78 citations · 300 across the 10 of their papers we have counts for
10 papers
Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation
Jian Zhang, Yuanqing Zhang, Huan Fu +6
Neural Radiance Fields (NeRF) have emerged as a potent paradigm for representing scenes and synthesizing photo-realistic images. A main limitation of conventional NeRFs is that the…
PipeMare: Asynchronous Pipeline Parallel DNN Training
Bowen Yang, Jian Zhang, Jonathan Li +3
Pipeline parallelism (PP) when training neural networks enables larger models to be partitioned spatially, leading to both lower network communication and overall higher hardware u…
On the Downstream Performance of Compressed Word Embeddings
Avner May, Jian Zhang, Tri Dao +1
Compressing word embeddings is important for deploying NLP models in memory-constrained settings. However, understanding what makes compressed embeddings perform well on downstream…
Low-Memory Neural Network Training: A Technical Report
Nimit S. Sohoni, Christopher R. Aberger, Megan Leszczynski +2
Memory is increasingly often the bottleneck when training neural network models. Despite this, techniques to lower the overall memory requirements of training have been less widely…
Low-Precision Random Fourier Features for Memory-Constrained Kernel Approximation
Jian Zhang, Avner May, Tri Dao +1
We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximatio…
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark
Cody Coleman, Daniel Kang, Deepak Narayanan +7
Researchers have proposed hardware, software, and algorithmic optimizations to improve the computational performance of deep learning. While some of these optimizations perform the…
High-Accuracy Low-Precision Training
Christopher De Sa, Megan Leszczynski, Jian Zhang +4
Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has b…
Deep Learning at 15PF: Supervised and Semi-Supervised Classification for Scientific Data
Thorsten Kurth, Jian Zhang, Nadathur Satish +12
This paper presents the first, 15-PetaFLOP Deep Learning system for solving scientific pattern classification problems on contemporary HPC architectures. We develop supervised conv…
YellowFin and the Art of Momentum Tuning
Jian Zhang, Ioannis Mitliagkas
Hyperparameter tuning is one of the most time-consuming workloads in deep learning. State-of-the-art optimizers, such as AdaGrad, RMSProp and Adam, reduce this labor by adaptively…
Parallel SGD: When does averaging help?
Jian Zhang, Christopher De Sa, Ioannis Mitliagkas +1
Consider a number of workers running SGD independently on the same pool of data and averaging the models every once in a while -- a common but not well understood practice. We stud…