9 papers
A Theory of Feature Learning in Kernel Models
Yunlu Chen, Yang Li, Keli Liu +1
We study feature learning in a compositional variant of kernel ridge regression in which the predictor is applied to a learnable linear transformation of the input. When the respon…
RAPTOR: Ridge-Adaptive Logistic Probes
Ziqi Gao, Yaotian Zhu, Qingcheng Zeng +4
Probing studies what information is encoded in a frozen LLM's layer representations by training a lightweight predictor on top of them. Beyond analysis, probes are often used opera…
A Compositional Kernel Model for Feature Learning
Feng Ruan, Keli Liu, Michael Jordan
We study a compositional variant of kernel ridge regression in which the predictor is applied to a coordinate-wise reweighting of the inputs. Formulated as a variational problem, t…
Robust Detection of Watermarks for Large Language Models Under Human Edits
Xiang Li, Feng Ruan, Huiyuan Wang +2
Watermarking has offered an effective approach to distinguishing text generated by large language models (LLMs) from human-written text. However, the pervasive presence of human ed…
A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules
Xiang Li, Feng Ruan, Huiyuan Wang +2
Since ChatGPT was introduced in November 2022, embedding (nearly) unnoticeable statistical signals into text generated by large language models (LLMs), also known as watermarking,…
On the Uniform Convergence of Subdifferentials in Stochastic Optimization and Learning
Feng Ruan
We investigate the uniform convergence of subdifferential mappings from empirical risk to population risk in nonsmooth, nonconvex stochastic optimization. This question is key to u…