36 citations · 39 across the 15 of their papers we have counts for
6 papers · 1 filter
Non-Asymptotic Analysis of Efficiency in Conformalized Regression
Yunzhen Yao, Lie He, Michael Gastpar
Conformal prediction provides prediction sets with coverage guarantees. The informativeness of conformal prediction depends on its efficiency, typically quantified by the expected…
The Conditional Regret-Capacity Theorem for Batch Universal Prediction
Marco Bondaschi, Michael Gastpar
We derive a conditional version of the classical regret-capacity theorem. This result can be used in universal prediction to find lower bounds on the minimal batch regret, which is…
What One Cannot, Two Can: Two-Layer Transformers Provably Represent Induction Heads on Any-Order Markov Chains
Chanakya Ekbote, Marco Bondaschi, Nived Rajaraman +4
In-context learning (ICL) is a hallmark capability of transformers, through which trained models learn to adapt to new tasks by leveraging information from the input context. Prior…
zip2zip: Inference-Time Adaptive Tokenization via Online Compression
Saibo Geng, Nathan Ranchin, Yunzhen yao +4
Tokenization efficiency plays a critical role in the performance and cost of large language models (LLMs), yet most models rely on static tokenizers optimized on general-purpose co…
From Markov to Laplace: How Mamba In-Context Learns Markov Chains
Marco Bondaschi, Nived Rajaraman, Xiuying Wei +5
While transformer-based language models have driven the AI revolution thus far, their computational complexity has spurred growing interest in viable alternatives, such as structur…
Leveraging Sparsity for Sample-Efficient Preference Learning: A Theoretical Perspective
Yunzhen Yao, Lie He, Michael Gastpar
This paper considers the sample-efficiency of preference learning, which models and predicts human choices based on comparative judgments. The minimax optimal estimation error rate…