5 papers
On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery
Renpu Liu, Ruida Zhou, Cong Shen +1
An intriguing property of the Transformer is its ability to perform in-context learning (ICL), where the Transformer can solve different inference tasks without parameter updating…
Weakly Private Information Retrieval from Heterogeneously Trusted Servers
Wenyuan Zhao, Yu-Shin Huang, Ruida Zhou +1
We study the problem of weakly private information retrieval (PIR) when there is heterogeneity in servers' trustworthiness under the maximal leakage (Max-L) metric and mutual infor…
On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures
Wei Shen, Ruida Zhou, Jing Yang +1
Although transformers have demonstrated impressive capabilities for in-context learning (ICL) in practice, theoretical understanding of the underlying mechanism that allows transfo…
Cost-Aware Optimal Pairwise Pure Exploration
Di Wu, Chengshuai Shi, Ruida Zhou +1
Pure exploration is one of the fundamental problems in multi-armed bandits (MAB). However, existing works mostly focus on specific pure exploration tasks, without a holistic view o…
Data-adaptive Differentially Private Prompt Synthesis for In-Context Learning
Fengyu Gao, Ruida Zhou, Tianhao Wang +2
Large Language Models (LLMs) rely on the contextual information embedded in examples/demonstrations to perform in-context learning (ICL). To mitigate the risk of LLMs potentially l…