5 papers
MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers
Linrui Ma, Chun Hei Lo, Xinyu Wang +12
The quadratic computational cost of traditional attention mechanisms poses a major bottleneck to the scalability and practical deployment of large language models (LLMs), particula…
Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks
Mengyu Zheng, Kai Han, Boxun Li +13
General-purpose agents such as OpenClaw are increasingly used as autonomous tool users, but their coding ability is difficult to measure under SWE-bench: a generic agent does not b…
Mask Is What DLLM Needs: A Masked Data Training Paradigm for Diffusion LLMs
Linrui Ma, Yufei Cui, Kai Han +1
Discrete diffusion models offer global context awareness and flexible parallel generation. However, uniform random noise schedulers in standard DLLM training overlook the highly no…
Diffusion In Diffusion: Reclaiming Global Coherence in Semi-Autoregressive Diffusion
Linrui Ma, Yufei Cui, Kai Han +1
One of the most compelling features of global discrete diffusion language models is their global bidirectional contextual capability. However, existing block-based diffusion studie…
Resona: Improving Context Copying in Linear Recurrence Models with Retrieval
Xinyu Wang, Linrui Ma, Jerry Huang +5
Recent shifts in the space of large language model (LLM) research have shown an increasing focus on novel architectures to compete with prototypical Transformer-based models that h…