3 citations · 3 across the 31 of their papers we have counts for
33 papers
Prefilling-dLLM v2: Learned Chunk Routing for Long-Context Diffusion Language Models
Qi Han, Jing Xiong, Ziyuan Zhang +1
Diffusion language models apply bidirectional attention over the whole context at every denoising step, so a longer context makes each step substantially more expensive. Sparsifyin…
Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance
Hengyuan Zhang, Chenming Shang, Zunhai Su +10
Best-of- is a widely used inference strategy for complex reasoning, whose effectiveness depends on whether sampled candidates can cover diverse and high-quality reasoning paths.…
Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus
Zunhai Su, Bohan Sun, Xialie Zhuang +9
We present the first systematic study of massive activations (MAs) in layer-interleaved Hybrid linear attention large language models (HLA LLMs), examining their architectural orga…
What Makes Interaction Trajectories Effective for Training Terminal Agents?
Sidi Yang, Chaofan Tao, Jierun Chen +11
Stronger code agents are commonly assumed to be superior teachers for post-training, yet this assumption remains poorly disentangled from task difficulty, harness design, and stude…
CktFormalizer: Autoformalization of Natural Language into Circuit Representations
Jing Xiong, Qi Han, Chenchen Ding +5
Hardware infrastructure is a critical bottleneck for LLM-driven circuit design, limiting what agents can express, compile, and iteratively refine within an agentic loop. To address…
CodeComp: Structural KV Cache Compression for Agentic Coding
Qiujiang Chen, Jing Xiong, Chenyang Zhao +2
Agentic code tasks such as fault localization and patch generation require processing long codebases under tight memory constraints, where the Key-Value (KV) cache becomes the prim…
Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation
Zunhai Su, Hengyuan Zhang, Wei Wu +24
As the foundational architecture of modern machine learning, Transformers have driven remarkable progress across diverse AI domains. Despite their transformative impact, a persiste…
Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidance
Xinrong Chen, Xu Chu, Yingmin Qiu +8
Large Vision-Language Models (LVLMs) can reason from image-text inputs and perform well in various multimodal tasks. Despite this success, they are affected by language priors and…
Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity
Hengyuan Zhang, Xinrong Chen, Zunhai Su +10
Layer-wise mixed-precision quantization (LMPQ) enables effective compression under extreme low-bit settings by allocating higher precision to sensitive layers. However, existing me…
MMSpec: Benchmarking Speculative Decoding for Vision-Language Models
Hui Shen, Xin Wang, Ping Zhang +11
Vision-language models (VLMs) achieve strong performance on multimodal tasks but suffer from high inference latency due to large model sizes and long multimodal contexts. Speculati…
MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generation
Ruiyao Liu, Hui Shen, Ping Zhang +16
Modern generative models have demonstrated the ability to solve challenging mathematical problems. In many real-world settings, however, mathematical solutions must be expressed vi…
V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
Dongyang Chen, Chaoyang Wang, Dezhao Su +6
Multimodal Large Language Models (MLLMs) have recently been applied to universal multimodal retrieval, where Chain-of-Thought (CoT) reasoning improves candidate reranking. However,…
BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
Junyu Chen, Jungang Li, Jing Xiong +11
Large language model inference is often bounded by memory footprint and bandwidth in resource-constrained deployments, making quantization fundamental to efficient serving. While p…
OVD: On-policy Verbal Distillation
Jing Xiong, Hui Shen, Shansan Gong +7
Knowledge distillation transfers reasoning capabilities from large teachers to efficient students. However, token-level on-policy distillation (OPD) constrains student exploration…
MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents
Peizhou Huang, Zixuan Zhong, Zhongwei Wan +12
Deep Research Agents (DRAs) generate citation-rich reports via multi-step search and synthesis, yet existing benchmarks mainly target text-only settings or short-form multimodal QA…
MMFormalizer: Multimodal Autoformalization in the Wild
Jing Xiong, Qi Han, Yunta Hsieh +11
Autoformalization, which translates natural language mathematics into formal statements to enable machine reasoning, faces fundamental challenges in the wild due to the multimodal…
APT-ClaritySet: A Large-Scale, High-Fidelity Labeled Dataset for APT Malware with Alias Normalization and Graph-Based Deduplication
Zhenhao Yin, Hanbing Yan, Huishu Lu +4
Large-scale, standardized datasets for Advanced Persistent Threat (APT) research are scarce, and inconsistent actor aliases and redundant samples hinder reproducibility. This paper…
AnchorTP: Resilient LLM Inference with State-Preserving Elastic Tensor Parallelism
Wendong Xu, Chujie Chen, He Xiao +8
Large Language Model (LLM) inference services demand exceptionally high availability and low latency, yet multi-GPU Tensor Parallelism (TP) makes them vulnerable to single-GPU fail…
DoPE: Denoising Rotary Position Embedding
Jing Xiong, Liyang Fan, Hui Shen +4
Positional encoding is essential for large language models (LLMs) to represent sequence order, yet recent studies show that Rotary Position Embedding (RoPE) can induce massive acti…
Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation
Hengyuan Zhang, Shiping Yang, Xiao Liang +8
Training student models on synthetic data generated by strong teacher models is a promising way to distilling the capabilities of teachers. However, recent studies show that strong…
ATTS: Asynchronous Test-Time Scaling via Conformal Prediction
Jing Xiong, Qiujiang Chen, Fanghua Ye +11
Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency. Speculative decoding is a natural way to accelerate the scaling proces…
LongEmotion: Measuring Emotional Intelligence of Large Language Models in Long-Context Interaction
Weichu Liu, Jing Xiong, Yuxuan Hu +10
Large language models (LLMs) have made significant progress in Emotional Intelligence (EI) and long-context modeling. However, existing benchmarks often overlook the fact that emot…
Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTS
Alex ZH Dou, Zhongwei Wan, Dongfei Cui +6
Test-time scaling has emerged as a promising paradigm in language modeling, leveraging additional computational resources at inference time to enhance model performance. In this wo…
From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes
Tianxu Wang, Zhuofan Zhang, Ziyu Zhu +5
3D visual grounding has made notable progress in localizing objects within complex 3D scenes. However, grounding referring expressions beyond objects in 3D scenes remains unexplore…