29 citations · 54 across the 9 of their papers we have counts for
11 papers
Muse Glimmer 30B: An On-Device Agentic Model
Arash Ebrahim Abtahizadeh, Muna Aghamelu, Adil Ahmed +327
We introduce Muse Glimmer, an open-source 30B parameter causal language model with perception encoder, purpose-built for autonomous agentic tasks on consumer hardware. Muse Glimmer…
AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
Yun He, Wenzhe Li, Hejia Zhang +22
Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, a…
The N-Grammys: Accelerating Autoregressive Inference with Learning-Free Batched Speculation
Lawrence Stewart, Matthew Trager, Sujan Kumar Gonugondla +1
Speculative decoding aims to speed up autoregressive generation of a language model by verifying in parallel the tokens generated by a smaller draft model.In this work, we explore…
Approximately Aligned Decoding
Daniel Melcer, Sujan Gonugondla, Pramuditha Perera +7
It is common to reject undesired outputs of Large Language Models (LLMs); however, current methods to do so require an excessive amount of computation to re-sample after a rejectio…
BASS: Batched Attention-optimized Speculative Sampling
Haifeng Qian, Sujan Kumar Gonugondla, Sungsoo Ha +6
Speculative decoding has emerged as a powerful method to improve latency and throughput in hosting large language models. However, most existing implementations focus on generating…
Token Alignment via Character Matching for Subword Completion
Ben Athiwaratkun, Shiqi Wang, Mingyue Shang +7
Generative models, widely utilized in various applications, can often struggle with prompts corresponding to partial tokens. This struggle stems from tokenization, where partial to…
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs
Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda +10
This study introduces bifurcated attention, a method designed to enhance language model inference in shared-context batch decoding scenarios. Our approach addresses the challenge o…
Greener yet Powerful: Taming Large Code Generation Models with Quantization
Xiaokai Wei, Sujan Gonugondla, Wasi Ahmad +13
ML-powered code generation aims to assist developers to write code in a more productive manner, by intelligently generating code blocks based on natural language prompts. Recently,…
Multi-lingual Evaluation of Code Generation Models
Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang +22
We present new benchmarks on evaluation code generation models: MBXP and Multilingual HumanEval, and MathQA-X. These datasets cover over 10 programming languages and are generated…
Fundamental Limits on Energy-Delay-Accuracy of In-memory Architectures in Inference Applications
Sujan Kumar Gonugondla, Charbel Sakr, Hassan Dbouk +1
This paper obtains fundamental limits on the computational precision of in-memory computing architectures (IMCs). An IMC noise model and associated SNR metrics are defined and thei…
A 481pJ/decision 3.4M decision/s Multifunctional Deep In-memory Inference Processor using Standard 6T SRAM Array
Mingu Kang, Sujan Gonugondla, Ameya Patil +1
This paper describes a multi-functional deep in-memory processor for inference applications. Deep in-memory processing is achieved by embedding pitch-matched low-SNR analog process…