355 citations · 980 across the 44 of their papers we have counts for
5 papers · 1 filter
AcceLLM: Accelerating LLM Inference using Redundancy for Load Balancing and Data Locality
Ilias Bournias, Lukas Cavigelli, Georgios Zacharopoulos
Large Language Model (LLM) inference on large-scale systems is expected to dominate future cloud infrastructures. Efficient LLM inference in cloud environments with numerous AI acc…
SSSD: Simply-Scalable Speculative Decoding
Michele Marzollo, Jiawei Zhuang, Niklas Roemer +3
Speculative Decoding has emerged as a popular technique for accelerating inference in Large Language Models. However, most existing approaches yield only modest improvements in pro…
Ascend-CC: Confidential Computing on Heterogeneous NPU for Emerging Generative AI Workloads
Aritra Dhar, Clément Thorens, Lara Magdalena Lazier +1
Cloud workloads have dominated generative AI based on large language models (LLM). Specialized hardware accelerators, such as GPUs, NPUs, and TPUs, play a key role in AI adoption d…
On-Device Domain Learning for Keyword Spotting on Low-Power Extreme Edge Embedded Systems
Cristian Cioflan, Lukas Cavigelli, Manuele Rusci +2
Keyword spotting accuracy degrades when neural networks are exposed to noisy environments. On-site adaptation to previously unseen noise is crucial to recovering accuracy loss, and…
Boosting keyword spotting through on-device learnable user speech characteristics
Cristian Cioflan, Lukas Cavigelli, Luca Benini
Keyword spotting systems for always-on TinyML-constrained applications require on-site tuning to boost the accuracy of offline trained classifiers when deployed in unseen inference…