17 papers
LLM Reasoning for Subjective Tasks: Failure Modes, Mitigation, and Dynamic Reasoning Routing
Juncheng Dong, Ding Tong, Ishan Gupta +1
Recommendation systems thrive on personalization, where ''correctness'' is rarely a binary truth but a matter of subjective human preference. As Large Language Models (LLMs) are de…
One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning
Bowen He, Juncheng Dong, Lin Lin +1
A central challenge in reinforcement learning (RL) is to learn models that generalize beyond the tasks on which they are trained, a goal traditionally pursued through multi-task an…
Rethinking Token Prediction: Tree-Structured Diffusion Language Model
Zihao Wu, Haoming Yang, Juncheng Dong +1
Discrete diffusion language models have emerged as a competitive alternative to auto-regressive language models, but training them efficiently under limited parameter and memory bu…
Boosting In-Context Learning in LLMs Through the Lens of Classical Supervised Learning
Korel Gundem, Juncheng Dong, Dennis Zhang +2
In-Context Learning (ICL) allows Large Language Models (LLMs) to adapt to new tasks with just a few examples, but their predictions often suffer from systematic biases, leading to…
Learning in Context, Guided by Choice: A Reward-Free Paradigm for Reinforcement Learning with Transformers
Juncheng Dong, Bowen He, Moyang Guo +3
In-context reinforcement learning (ICRL) leverages the in-context learning capabilities of transformer models (TMs) to efficiently generalize to unseen sequential decision-making t…
In-Context Reinforcement Learning From Suboptimal Historical Data
Juncheng Dong, Moyang Guo, Ethan X. Fang +2
Transformer models have achieved remarkable empirical successes, largely due to their in-context learning capabilities. Inspired by this, we explore training an autoregressive tran…