6 papers
A Unified Framework for In-Context Learning with Causal and Masked Language Models
Chenrui Liu, Chuanlong Xie, Falong Tan +2
In-context learning (ICL) has emerged as a central capability of pretrained language models, yet its theoretical analysis has focused primarily on causal language models trained by…
Asymptotic Distribution-Free Tests for Ultra-high Dimensional Parametric Regressions via Projected Empirical Processes and -value Combination
Falong Tan, Shan Tang, Lixing Zhu
This paper develops a novel methodology for testing the goodness-of-fit of sparse parametric regression models based on projected empirical processes and p-value combination, where…
A Two-Step Projection-Based Goodness-of-Fit Test for Ultra-High Dimensional Sparse Regressions
Falong Tan, Jie Liu, Heng Peng +1
This paper proposes a novel two-step strategy for testing the goodness-of-fit of parametric regression models in ultra-high dimensional sparse settings, where the predictor dimensi…
Variable Selection for Multi-Source Count Data with Controlled False Discovery Rate
Shan Tang, Shanjun Mao, Shourong Ma +1
The rapid generation of complex, highly skewed, and zero-inflated multi-source count data poses significant challenges for variable selection, particularly in biomedical domains li…
Weighted residual empirical processes, martingale transformations, and model specification tests for regressions with diverging number of parameters
Falong Tan, Xu Guo, Lixing Zhu
This paper explores hypothesis testing for the parametric forms of the mean and variance functions in regression models under diverging-dimension settings. To mitigate the curse of…
In-Context Learning as Nonparametric Conditional Probability Estimation: Risk Bounds and Optimality
Chenrui Liu, Falong Tan, Chuanlong Xie +2
This paper investigates the expected excess risk of in-context learning (ICL) for multiclass classification. We formalize each task as a sequence of labeled examples followed by a…