19 citations · 24 across the 5 of their papers we have counts for
5 papers
Sparser, Faster, Lighter Transformer Language Models
Edoardo Cetin, Stefano Peluchetti, Emilio Castillo +3
Scaling autoregressive large language models (LLMs) has driven unprecedented progress but comes with vast computational costs. In this work, we tackle these costs by leveraging uns…
CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs
Hiroyuki Ootomo, Akira Naruse, Corey Nolet +3
Approximate Nearest Neighbor Search (ANNS) plays a critical role in various disciplines spanning data mining and artificial intelligence, from information retrieval and computer vi…
Custom 8-bit floating point value format for reducing shared memory bank conflict in approximate nearest neighbor search
Hiroyuki Ootomo, Akira Naruse
The k-nearest neighbor search is used in various applications such as machine learning, computer vision, database search, and information retrieval. While the computational cost of…
Scalable and Practical Natural Gradient for Large-Scale Deep Learning
Kazuki Osawa, Yohei Tsuji, Yuichiro Ueno +3
Large-scale distributed training of deep neural networks results in models with worse generalization performance as a result of the increase in the effective mini-batch size. Previ…
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
Kazuki Osawa, Yohei Tsuji, Yuichiro Ueno +3
Large-scale distributed training of deep neural networks suffer from the generalization gap caused by the increase in the effective mini-batch size. Previous approaches try to solv…