Jing Xiong

Prefilling-dLLM v2: Learned Chunk Routing for Long-Context Diffusion Language Models

Qi Han, Jing Xiong, Ziyuan Zhang +1

Diffusion language models apply bidirectional attention over the whole context at every denoising step, so a longer context makes each step substantially more expensive. Sparsifyin…

Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance

Hengyuan Zhang, Chenming Shang, Zunhai Su +10

Best-of- $N$ is a widely used inference strategy for complex reasoning, whose effectiveness depends on whether sampled candidates can cover diverse and high-quality reasoning paths.…

Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus

Zunhai Su, Bohan Sun, Xialie Zhuang +9

We present the first systematic study of massive activations (MAs) in layer-interleaved Hybrid linear attention large language models (HLA LLMs), examining their architectural orga…

CktFormalizer: Autoformalization of Natural Language into Circuit Representations

Jing Xiong, Qi Han, Chenchen Ding +5

Hardware infrastructure is a critical bottleneck for LLM-driven circuit design, limiting what agents can express, compile, and iteratively refine within an agentic loop. To address…

CodeComp: Structural KV Cache Compression for Agentic Coding

Qiujiang Chen, Jing Xiong, Chenyang Zhao +2

Agentic code tasks such as fault localization and patch generation require processing long codebases under tight memory constraints, where the Key-Value (KV) cache becomes the prim…

OVD: On-policy Verbal Distillation

Jing Xiong, Hui Shen, Shansan Gong +7

Knowledge distillation transfers reasoning capabilities from large teachers to efficient students. However, token-level on-policy distillation (OPD) constrains student exploration…

MMFormalizer: Multimodal Autoformalization in the Wild

Jing Xiong, Qi Han, Yunta Hsieh +11

Autoformalization, which translates natural language mathematics into formal statements to enable machine reasoning, faces fundamental challenges in the wild due to the multimodal…

DoPE: Denoising Rotary Position Embedding

Jing Xiong, Liyang Fan, Hui Shen +4

Positional encoding is essential for large language models (LLMs) to represent sequence order, yet recent studies show that Rotary Position Embedding (RoPE) can induce massive acti…

ATTS: Asynchronous Test-Time Scaling via Conformal Prediction

Jing Xiong, Qiujiang Chen, Fanghua Ye +11

Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency. Speculative decoding is a natural way to accelerate the scaling proces…

LongEmotion: Measuring Emotional Intelligence of Large Language Models in Long-Context Interaction

Weichu Liu, Jing Xiong, Yuxuan Hu +10

Large language models (LLMs) have made significant progress in Emotional Intelligence (EI) and long-context modeling. However, existing benchmarks often overlook the fact that emot…

Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTS

Alex ZH Dou, Zhongwei Wan, Dongfei Cui +6

Test-time scaling has emerged as a promising paradigm in language modeling, leveraging additional computational resources at inference time to enhance model performance. In this wo…

SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning

Zhongwei Wan, Zhihao Dou, Che Liu +11

Multimodal large language models (MLLMs) have shown promising capabilities in reasoning tasks, yet still struggle with complex problems requiring explicit self-reflection and self-…

SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving

Wendong Xu, Jing Xiong, Chenyang Zhao +16

We present SwingArena, a competitive evaluation framework for Large Language Models (LLMs) that closely mirrors real-world software development workflows. Unlike traditional static…

ParallelComp: Parallel Long-Context Compressor for Length Extrapolation

Jing Xiong, Jianghan Shen, Chuanyang Zheng +7

Extrapolating ultra-long contexts (text length >128K) remains a major challenge for large language models (LLMs), as most training-free extrapolation methods are not only severely…

LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning

Yansheng Mao, Jiaqi Li, Fanxu Meng +3

Long context understanding remains challenging for large language models due to their limited context windows. This paper introduces Long Input Fine-Tuning (LIFT) for long context…

UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation

Zixuan Li, Jing Xiong, Fanghua Ye +9

We present UncertaintyRAG, a novel approach for long-context Retrieval-Augmented Generation (RAG) that utilizes Signal-to-Noise Ratio (SNR)-based span uncertainty to estimate simil…

UNComp: Can Matrix Entropy Uncover Sparsity? -- A Compressor Design from an Uncertainty-Aware Perspective

Jing Xiong, Jianghan Shen, Fanghua Ye +9

Deploying large language models (LLMs) for long-context inference remains challenging due to their substantial memory and computational demands. While techniques such as Key-Value…