3 papers
cs.LG2026
Learnable Low-Rank Polynomial Sketch for Effective Linear Attention
Zongyue Qin, Haoran Deng, Jason Cong +1
Softmax attention in Transformers suffers from quadratic complexity in sequence length, making it impractical for long-context applications. Linear attention alleviates this issue…
cs.CL2025
Beyond Length: Quantifying Long-Range Information for Long-Context LLM Pretraining Data
Haoran Deng, Yingyu Lin, Zhenghao Lin +4
Long-context language models unlock advanced capabilities in reasoning, code generation, and document summarization by leveraging dependencies across extended spans of text. Howeve…
cs.LG2024
A Skewness-Based Criterion for Addressing Heteroscedastic Noise in Causal Discovery
Yingyu Lin, Yuxing Huang, Wenqin Liu +6
Real-world data often violates the equal-variance assumption (homoscedasticity), making it essential to account for heteroscedastic noise in causal discovery. In this work, we expl…