571 citations · 858 across the 10 of their papers we have counts for
10 papers
Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale
Zhaoxia, Deng, Jongsoo Park +17
Tremendous success of machine learning (ML) and the unabated growth in ML model complexity motivated many ML-specific designs in both CPU and accelerator architectures to speed up…
Efficient Soft-Error Detection for Low-precision Deep Learning Recommendation Models
Sihuan Li, Jianyu Huang, Ping Tak Peter Tang +4
Soft error, namely silent corruption of signal or datum in a computer system, cannot be caverlierly ignored as compute and communication density grow exponentially. Soft error dete…
Mixed-Precision Embedding Using a Cache
Jie Amy Yang, Jianyu Huang, Jongsoo Park +2
In recommendation systems, practitioners observed that increase in the number of embedding tables and their sizes often leads to significant improvement in model performances. Give…
Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations
Greg Henry, Ping Tak Peter Tang, Alexander Heinecke
In recent years fused-multiply-add (FMA) units with lower-precision multiplications and higher-precision accumulation have proven useful in machine learning/artificial intelligence…
Dictionary Learning by Dynamical Neural Networks
Tsung-Han Lin, Ping Tak Peter Tang
A dynamical neural network consists of a set of interconnected neurons that interact over time continuously. It can exhibit computational properties in the sense that the dynamical…
A Progressive Batching L-BFGS Method for Machine Learning
Raghu Bollapragada, Dheevatsa Mudigere, Jorge Nocedal +2
The standard L-BFGS method relies on gradient approximations that are not dominated by noise, so that search directions are descent directions, the line search is reliable, and qua…
Sparse Coding by Spiking Neural Networks: Convergence Theory and Computational Results
Ping Tak Peter Tang, Tsung-Han Lin, Mike Davies
In a spiking neural network (SNN), individual neurons operate autonomously and only communicate with other neurons sparingly and asynchronously via spike signals. These characteris…
Enabling Sparse Winograd Convolution by Native Pruning
Sheng Li, Jongsoo Park, Ping Tak Peter Tang
Sparse methods and the use of Winograd convolutions are two orthogonal approaches, each of which significantly accelerates convolution computations in modern CNNs. Sparse Winograd…
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal +2
The stochastic gradient descent (SGD) method and its variants are algorithms of choice for many Deep Learning tasks. These methods operate in a small-batch regime wherein a fractio…
Faster CNNs with Direct Sparse Convolutions and Guided Pruning
Jongsoo Park, Sheng Li, Wei Wen +4
Phenomenally successful in practical inference problems, convolutional neural networks (CNN) are widely deployed in mobile devices, data centers, and even supercomputers. The numbe…