7 papers
Prefilling-dLLM v2: Learned Chunk Routing for Long-Context Diffusion Language Models
Qi Han, Jing Xiong, Ziyuan Zhang +1
Diffusion language models apply bidirectional attention over the whole context at every denoising step, so a longer context makes each step substantially more expensive. Sparsifyin…
QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching
Zunhai Su, Yuxuan Sun, Jianchao Tan +5
Multi-Head Latent Attention (MLA) enables expressive multi-head attention with compact caches for its content and decoupled RoPE paths, yet cache memory still scales linearly with…
Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance
Hengyuan Zhang, Chenming Shang, Zunhai Su +10
Best-of- is a widely used inference strategy for complex reasoning, whose effectiveness depends on whether sampled candidates can cover diverse and high-quality reasoning paths.…
CktFormalizer: Autoformalization of Natural Language into Circuit Representations
Jing Xiong, Qi Han, Chenchen Ding +5
Hardware infrastructure is a critical bottleneck for LLM-driven circuit design, limiting what agents can express, compile, and iteratively refine within an agentic loop. To address…
XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression
Zunhai Su, Weihao Ye, Hansen Feng +5
Learning-based 3D visual geometry models have significantly advanced with the advent of large-scale transformers. Among these, StreamVGGT leverages frame-wise causal attention to d…
XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression
Zunhai Su, Weihao Ye, Hansen Feng +5
Learning-based 3D visual geometry models have benefited substantially from large-scale transformers. Among these, StreamVGGT leverages frame-wise causal attention for strong stream…
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
Hengyuan Zhang, Zhihao Zhang, Mingyang Wang +26
Mechanistic Interpretability (MI) has emerged as a vital approach to demystify the opaque decision-making of Large Language Models (LLMs). However, existing reviews primarily treat…