7 citations · 9 across the 8 of their papers we have counts for
8 papers
OverFill: Two-Stage Models for Efficient Language Model Decoding
Woojeong Kim, Junxiong Wang, Jing Nathan Yan +2
Large language models (LLMs) excel across diverse tasks but face significant deployment challenges due to high inference costs. LLM inference comprises prefill (compute-bound) and…
M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
Junxiong Wang, Wen-Ding Li, Daniele Paliotta +3
Effective reasoning is crucial to solving complex mathematical problems. Recent large language models (LLMs) have boosted performance by scaling test-time computation through long…
The Mamba in the Llama: Distilling and Accelerating Hybrid Models
Junxiong Wang, Daniele Paliotta, Avner May +2
Linear RNN architectures, like Mamba, can be competitive with Transformer models in language modeling while having advantageous deployment characteristics. Given the focus on train…
Entity Disambiguation via Fusion Entity Decoding
Junxiong Wang, Ali Mousavi, Omar Attia +5
Entity disambiguation (ED), which links the mentions of ambiguous entities to their referent entities in a knowledge base, serves as a core component in entity linking (EL). Existi…
MambaByte: Token-free Selective State Space Model
Junxiong Wang, Tushaar Gangavarapu, Jing Nathan Yan +1
Token-free language models learn directly from raw bytes and remove the inductive bias of subword tokenization. Operating on bytes, however, results in significantly longer sequenc…
Diffusion Models Without Attention
Jing Nathan Yan, Jiatao Gu, Alexander M. Rush
In recent advancements in high-fidelity image generation, Denoising Diffusion Probabilistic Models (DDPMs) have emerged as a key player. However, their application at high resoluti…
Predicting Text Preference Via Structured Comparative Reasoning
Jing Nathan Yan, Tianqi Liu, Justin T Chiu +9
Comparative reasoning plays a crucial role in text preference prediction; however, large language models (LLMs) often demonstrate inconsistencies in their reasoning. While approach…
Symbolic Planning and Code Generation for Grounded Dialogue
Justin T. Chiu, Wenting Zhao, Derek Chen +3
Large language models (LLMs) excel at processing and generating both text and code. However, LLMs have had limited applicability in grounded task-oriented dialogue as they are diff…