701 citations · 1.1k across the 18 of their papers we have counts for
3 papers · 1 filter
NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference
Joonsang Yu, Junki Park, Seongmin Park +4
Non-linear operations such as GELU, Layer normalization, and Softmax are essential yet costly building blocks of Transformer models. Several prior works simplified these operations…
Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling
Kyuhong Shim, Iksoo Choi, Wonyong Sung +1
While Transformer-based models have shown impressive language modeling performance, the large computation cost is often prohibitive for practical use. Attention head pruning, which…
Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead
Muhammad Shafique, Mahum Naseer, Theocharis Theocharides +4
Machine Learning (ML) techniques have been rapidly adopted by smart Cyber-Physical Systems (CPS) and Internet-of-Things (IoT) due to their powerful decision-making capabilities. Ho…