22 citations · 23 across the 4 of their papers we have counts for
4 papers
The N-Grammys: Accelerating Autoregressive Inference with Learning-Free Batched Speculation
Lawrence Stewart, Matthew Trager, Sujan Kumar Gonugondla +1
Speculative decoding aims to speed up autoregressive generation of a language model by verifying in parallel the tokens generated by a smaller draft model.In this work, we explore…
Token Alignment via Character Matching for Subword Completion
Ben Athiwaratkun, Shiqi Wang, Mingyue Shang +7
Generative models, widely utilized in various applications, can often struggle with prompts corresponding to partial tokens. This struggle stems from tokenization, where partial to…
Greener yet Powerful: Taming Large Code Generation Models with Quantization
Xiaokai Wei, Sujan Gonugondla, Wasi Ahmad +13
ML-powered code generation aims to assist developers to write code in a more productive manner, by intelligently generating code blocks based on natural language prompts. Recently,…
A 481pJ/decision 3.4M decision/s Multifunctional Deep In-memory Inference Processor using Standard 6T SRAM Array
Mingu Kang, Sujan Gonugondla, Ameya Patil +1
This paper describes a multi-functional deep in-memory processor for inference applications. Deep in-memory processing is achieved by embedding pitch-matched low-SNR analog process…