12 citations · 13 across the 6 of their papers we have counts for
7 papers
Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
Xingyu Zhu, Pu, Yi +6
Chunked KV-cache compression reduces the memory and attention costs of long-context inference by compressing windows of consecutive tokens into fewer cache entries at a fixed strid…
T^2MLR: Transformer with Temporal Middle-Layer Recurrence
Ziyang Cai, Xingyu Zhu, Yihe Dong +2
Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reason…
Contextual Drag: How Errors in the Context Affect LLM Reasoning
Yun Cheng, Xingyu Zhu, Haoyu Zhao +1
Central to many self-improvement pipelines for large language models (LLMs) is the assumption that models can improve by reflecting on past mistakes. We study a phenomenon termed c…
On the Power of Context-Enhanced Learning in LLMs
Xingyu Zhu, Abhishek Panigrahi, Sanjeev Arora
We formalize a new concept for LLMs, context-enhanced learning. It involves standard gradient-based learning on text except that the context is enhanced with additional data on whi…
Fairness in the Assignment Problem with Uncertain Priorities
Zeyu Shen, Zhiyi Wang, Xingyu Zhu +2
In the assignment problem, a set of items must be allocated to unit-demand agents who express ordinal preferences (rankings) over the items. In the assignment problem with prioriti…
Understanding Edge-of-Stability Training Dynamics with a Minimalist Example
Xingyu Zhu, Zixuan Wang, Xiang Wang +2
Recently, researchers observed that gradient descent for deep neural networks operates in an ``edge-of-stability'' (EoS) regime: the sharpness (maximum eigenvalue of the Hessian) i…
Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks
Yikai Wu, Xingyu Zhu, Chenwei Wu +2
Hessian captures important properties of the deep neural network loss landscape. Previous works have observed low rank structure in the Hessians of neural networks. In this paper,…