5 papers
Task Vectors in In-Context Learning: Emergence, Formation, and Benefit
Liu Yang, Ziqian Lin, Kangwook Lee +2
In-context learning is a remarkable capability of transformers, referring to their ability to adapt to specific tasks based on a short history or context. Previous research has fou…
Unifying Generative and Dense Retrieval for Sequential Recommendation
Liu Yang, Fabian Paischer, Kaveh Hassani +11
Sequential dense retrieval models utilize advanced sequence learning techniques to compute item and user representations, which are then used to rank relevant items for a user thro…
An Empirical Study on the Power of Future Prediction in Partially Observable Environments
Jeongyeol Kwon, Liu Yang, Robert Nowak +1
Learning good representations of historical contexts is one of the core challenges of reinforcement learning (RL) in partially observable environments. While self-predictive auxili…
Looped Transformers are Better at Learning Learning Algorithms
Liu Yang, Kangwook Lee, Robert Nowak +1
Transformers have demonstrated effectiveness in in-context solving data-fitting problems from various (latent) models, as reported by Garg et al. However, the absence of an inheren…
On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation
Jeongyeol Kwon, Dohyun Kwon, Stephen Wright +1
In this work, we study first-order algorithms for solving Bilevel Optimization (BO) where the objective functions are smooth but possibly nonconvex in both levels and the variables…