219 citations · 284 across the 7 of their papers we have counts for
3 papers · 1 filter
Scaling On-Device GPU Inference for Large Generative Models
Jiuqiang Tang, Raman Sarokin, Ekaterina Ignasheva +5
Driven by the advancements in generative AI, large machine learning models have revolutionized domains such as image processing, audio synthesis, and speech recognition. While serv…
Efficient Memory Management for Deep Neural Net Inference
Yury Pisarchyk, Juhyun Lee
While deep neural net inference was considered a task for servers only, latest advances in technology allow the task of inference to be moved to mobile and embedded devices, desire…
On-Device Neural Net Inference with Mobile GPUs
Juhyun Lee, Nikolay Chirkov, Ekaterina Ignasheva +6
On-device inference of machine learning models for mobile phones is desirable due to its lower latency and increased privacy. Running such a compute-intensive task solely on the mo…