4 papers
A Framework for Quantifying How Pre-Training and Context Benefit In-Context Learning
Bingqing Song, Jiaxiang Li, Rong Wang +2
Pre-trained large language models have demonstrated a strong ability to learn from context, known as in-context learning (ICL). Despite a surge of recent applications that leverage…
Do LLMs Recognize Your Latent Preferences? A Benchmark for Latent Information Discovery in Personalized Interaction
Ioannis Tsaknakis, Bingqing Song, Shuyu Gan +5
Large Language Models (LLMs) excel at producing broadly relevant text, but this generality becomes a limitation when user-specific preferences are required, such as recommending re…
Effectively Steer LLM To Follow Preference via Building Confident Directions
Bingqing Song, Boran Han, Shuai Zhang +5
Having an LLM that aligns with human preferences is essential for accommodating individual needs, such as maintaining writing style or generating specific topics of interest. The m…
Unraveling the Gradient Descent Dynamics of Transformers
Bingqing Song, Boran Han, Shuai Zhang +2
While the Transformer architecture has achieved remarkable success across various domains, a thorough theoretical foundation explaining its optimization dynamics is yet to be fully…