30 citations · 98 across the 13 of their papers we have counts for
9 papers · 1 filter
XOR-Trellis: Ultra-Low-Complexity Dequantization and Curvature-Aware Hadamard-Free LLM Quantization
Xiaofan Que, Nir Elkayam, Spandan Pyakurel +2
Trellis-coded quantization enables high-dimensional compression of large language model (LLM) weights at ultra-low bit widths without the exponentially large codebooks required by…
Highly Optimized Kernels and Fine-Grained Codebooks for LLM Inference on Arm CPUs
Dibakar Gope, David Mansell, Danny Loh +1
Large language models (LLMs) have transformed the way we think about language understanding and generation, enthralling both researchers and developers. However, deploying LLMs for…
PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devices
Yuji Chai, Devashree Tripathy, Chuteng Zhou +6
The ability to accurately predict deep neural network (DNN) inference performance metrics, such as latency, power, and memory footprint, for an arbitrary DNN on a target hardware p…
High Throughput Matrix-Matrix Multiplication between Asymmetric Bit-Width Operands
Dibakar Gope, Jesse Beu, Matthew Mattina
Matrix multiplications between asymmetric bit-width operands, especially between 8- and 4-bit operands are likely to become a fundamental kernel of many important workloads includi…
Ternary MobileNets via Per-Layer Hybrid Filter Banks
Dibakar Gope, Jesse Beu, Urmish Thakker +1
MobileNets family of computer vision neural networks have fueled tremendous progress in the design and organization of resource-efficient architectures in recent years. New applica…
Pushing the limits of RNN Compression
Urmish Thakker, Igor Fedorov, Jesse Beu +4
Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size. As a result, there is a need for compression techniques that can signi…
Run-Time Efficient RNN Compression for Inference on Edge Devices
Urmish Thakker, Jesse Beu, Dibakar Gope +2
Recurrent neural networks can be large and compute-intensive, yet many applications that benefit from RNNs run on small devices with very limited compute and storage capabilities w…
Compressing RNNs for IoT devices by 15-38x using Kronecker Products
Urmish Thakker, Jesse Beu, Dibakar Gope +4
Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size.As a result, there is a need for compression techniques that can signif…
Ternary Hybrid Neural-Tree Networks for Highly Constrained IoT Applications
Dibakar Gope, Ganesh Dasika, Matthew Mattina
Machine learning-based applications are increasingly prevalent in IoT devices. The power and storage constraints of these devices make it particularly challenging to run modern neu…