21 papers
DD-RNO: A Domain-Decomposed Routed Neural Operator for Airfoil Flow Prediction
T. A. Mehta, P. S. Bhati, H. D. Akolekar
Deep learning surrogates for RANS flow prediction around airfoils face two persistent bottlenecks. A single neural architecture cannot simultaneously resolve sharp near-wall bounda…
Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory
Rubin Wei, Jiaqi Cao, Jiarui Wang +4
The paper presents Memory Decoder at Scale, a pretrained parametric long‑term memory module for decoder‑only language models that is scaled up to 6.9 B parameters and shown to impr…
MemSFT: Mitigating Alignment Tax with an External Parametric Memory
Jiarui Wang, Xiang Shi, Jiaqi Cao +8
Adapting Large Language Models (LLMs) to specialized domains often incurs an alignment tax, as fine-tuning on domain-specific tasks can cause catastrophic forgetting and substantia…
Information-Aware KV Cache Compression for Long Reasoning
Jushi Kai, Zhuiri Xiao, Alexandra Birch +1
Reasoning capability has advanced rapidly in large language models (LLMs), leading to an increasing size of key-value (KV) cache in both prefilling and decoding stages. Existing KV…
Depth-Attention: Cross-Layer Value Mixing for Language Models
Boyi Zeng, Yiqin Hao, Zitong Wang +7
Self-attention selects information freely across the sequence, but across depth, Transformers merely add each layer's output to the residual stream, so later layers cannot selectiv…
Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models
Huanyu Wang, Jushi Kai, Haoli Bai +4
Vision-Language Models (VLMs) incur substantial computational overhead and inference latency due to the large number of vision tokens introduced by high-resolution image and video…