2 citations · 2 across the 2 of their papers we have counts for
2 papers
cs.LG2023
Leveraging Speculative Sampling and KV-Cache Optimizations Together for Generative AI using OpenVINO
Haim Barad, Ekaterina Aidova, Yury Gorbachev
Inference optimizations are critical for improving user experience and reducing infrastructure costs and power consumption. In this article, we illustrate a form of dynamic executi…
cs.CV2020★ 2 cited
Neural Network Compression Framework for fast model inference
Alexander Kozlov, Ivan Lazarevich, Vasily Shamporov +2
In this work we present a new framework for neural networks compression with fine-tuning, which we called Neural Network Compression Framework (NNCF). It leverages recent advances…