2 papers
cs.LG2026
SpecQuant: Speculative Decoding with Multi-Parent Quantization for Adaptive LLM Inference
Harish KB, Jagadeeswaran M, Pradheep P +2
Running large language models (LLMs) locally continues to be limited by restrictions of compute and memory on consumer hardware. The popular acceleration technologies, such as quan…
cs.LG2026
FrugalSOT - Frugal Search Over the Models
Pradheep P, Yuvanesh S, Harish KB +4
In on-device NLP tasks, limited resources of embedded hardware, such as the Raspberry Pi 5, require efficient inference strategies. This paper introduces FrugalSOT (Frugal Search O…