7 citations · 8 across the 11 of their papers we have counts for
11 papers
Spend Your Rollouts Where It Counts: Rollout Allocation for Group-Based RL Post-Training
Woojeong Kim, Ziyi Yang, Jing Nathan Yan +1
Reinforcement learning (RL) is the dominant paradigm for post-training large language models. However, in the online, on-policy setting, rollout generation dominates the computatio…
The Efficiency Gap in Byte Modeling
Celine Lee, Jing Nathan Yan, Chen Liang +9
Modern language models have historically relied on two dominant design choices: subword tokenization and autoregressive (AR) ordering. These design decisions bake in priors that di…
OverFill: Two-Stage Models for Efficient Language Model Decoding
Woojeong Kim, Junxiong Wang, Jing Nathan Yan +2
Large language models (LLMs) excel across diverse tasks but face significant deployment challenges due to high inference costs. LLM inference comprises prefill (compute-bound) and…
CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward
Zhiqiang Wang, Pengbin Feng, Yanbin Lin +4
We propose Fuzzy Group Relative Policy Reward (FGRPR), a novel framework that integrates Group Relative Policy Optimization (GRPO) with a fuzzy reward function to enhance learning…
TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model
Cheng Yang, Yang Sui, Jinqi Xiao +8
Vision-Language Models (VLMs) demand substantial computational resources during inference, largely due to the extensive visual input tokens for representing visual information. Pre…
State Space Models are Strong Text Rerankers
Zhichao Xu, Jinghua Yan, Ashim Gupta +1
Transformers dominate NLP and IR; but their inference inefficiencies and challenges in extrapolating to longer contexts have sparked interest in alternative model architectures. Am…
A Controlled Study on Long Context Extension and Generalization in LLMs
Yi Lu, Jing Nathan Yan, Songlin Yang +6
Broad textual understanding and in-context learning require language models that utilize full document contexts. Due to the implementation challenges associated with directly train…
MambaByte: Token-free Selective State Space Model
Junxiong Wang, Tushaar Gangavarapu, Jing Nathan Yan +1
Token-free language models learn directly from raw bytes and remove the inductive bias of subword tokenization. Operating on bytes, however, results in significantly longer sequenc…
Diffusion Models Without Attention
Jing Nathan Yan, Jiatao Gu, Alexander M. Rush
In recent advancements in high-fidelity image generation, Denoising Diffusion Probabilistic Models (DDPMs) have emerged as a key player. However, their application at high resoluti…
Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning
Yue Yu, Jiaming Shen, Tianqi Liu +5
Large language models (LLMs) have shown remarkable capabilities in various natural language understanding tasks. With only a few demonstration examples, these LLMs can quickly adap…
Predicting Text Preference Via Structured Comparative Reasoning
Jing Nathan Yan, Tianqi Liu, Justin T Chiu +9
Comparative reasoning plays a crucial role in text preference prediction; however, large language models (LLMs) often demonstrate inconsistencies in their reasoning. While approach…