11 citations · 31 across the 16 of their papers we have counts for
6 papers · 1 filter
Transformers as Support Vector Machines
Davoud Ataee Tarzanagh, Yingcong Li, Christos Thrampoulidis +1
Since its inception in "Attention Is All You Need", transformer architecture has led to revolutionary advancements in NLP. The attention layer within the transformer admits a seque…
Max-Margin Token Selection in Attention Mechanism
Davoud Ataee Tarzanagh, Yingcong Li, Xuechen Zhang +1
Attention mechanism is a central component of the transformer architecture which led to the phenomenal success of large language models. However, the theoretical principles underly…
Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning
Yingcong Li, Kartik Sreenivasan, Angeliki Giannou +2
Chain-of-thought (CoT) is a method that enables language models to handle complex reasoning tasks by decomposing them into simpler steps. Despite its success, the underlying mechan…
Provable Pathways: Learning Multiple Tasks over Multiple Paths
Yingcong Li, Samet Oymak
Constructing useful representations across a large number of tasks is a key requirement for sample-efficient intelligent systems. A traditional idea in multitask learning (MTL) is…
Transformers as Algorithms: Generalization and Stability in In-context Learning
Yingcong Li, M. Emrullah Ildiz, Dimitris Papailiopoulos +1
In-context learning (ICL) is a type of prompting where a transformer model operates on a sequence of (input, output) examples and performs inference on-the-fly. In this work, we fo…
Stochastic Contextual Bandits with Long Horizon Rewards
Yuzhen Qin, Yingcong Li, Fabio Pasqualetti +2
The growing interest in complex decision-making and language modeling problems highlights the importance of sample-efficient learning over very long horizons. This work takes a ste…