Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
Accelerated AI Inference via Dynamic Execution Methods
Haim Barad, Jascha Achterberg, Tien Pei Chou +1
In this paper, we focus on Dynamic Execution techniques that optimize the computation flow based on input. This aims to identify simpler problems that can be solved using fewer res…
cs.LG2023
Leveraging Speculative Sampling and KV-Cache Optimizations Together for Generative AI using OpenVINO
Haim Barad, Ekaterina Aidova, Yury Gorbachev
Inference optimizations are critical for improving user experience and reducing infrastructure costs and power consumption. In this article, we illustrate a form of dynamic executi…