2 citations · 2 across the 17 of their papers we have counts for
7 papers · 1 filter
High-Dimensional Simulation-Based Inference in Latent Spaces
Lars Kühmichel, Stefan T. Radev, Bhanu Prasanna Koppolu +3
Neural simulation-based inference (SBI) has been widely successful in inferring a relatively small number of interpretable parameters from potentially high-dimensional observations…
ARM: Attention with Routed-Memory for Learnable Sparse Control
Qiuhao Zeng, Jerry Huang, Peng Lu +7
Despite advances in long-context inference, large language models (LLMs) remain fundamentally limited by the key-value (KV) caching mechanisms that are necessary for stable computa…
Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing
Xinyu Wang, Sicheng Lyu, Yu Gu +4
Model editing updates a pre-trained LLM with new facts or rules without retraining while preserving unrelated behavior. In real deployment, edits arrive as long streams, creating a…
Sublinear iterations can suffice even for DDPMs
Matthew S. Zhang, Stephen Huan, Jerry Huang +3
SDE-based methods such as denoising diffusion probabilistic models (DDPMs) have shown remarkable success in real-world sample generation tasks. Prior analyses of DDPMs have been fo…
Mamba Modulation: On the Length Generalization of Mamba
Peng Lu, Jerry Huang, Qiuhao Zeng +4
The quadratic complexity of the attention mechanism in Transformer models has motivated the development of alternative architectures with sub-quadratic scaling, such as state-space…
Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs
Behnoush Khavari, Mehran Shakerinava, Jayesh Khullar +4
Recent work has shown that LRNN models such as S4D, Mamba, and DeltaNet lack state-tracking capability due to either time-invariant transition matrices or restricted eigenvalue ran…
Calibrated Language Models and How to Find Them with Label Smoothing
Jerry Huang, Peng Lu, Qiuhao Zeng
Recent advances in natural language processing (NLP) have opened up greater opportunities to enable fine-tuned large language models (LLMs) to behave as more powerful interactive a…