19 papers
Magnetic fields in extreme primordial halos: turbulent collapse and implications for early quasar formation
V. B. Díaz, D. R. G. Schleicher, M. A. Latif +1
It is sometimes suggested that the most massive quasars at high redshift may have formed from rare high-sigma peaks in the cosmic density field. We explore here the evolution in a…
Decoding Rewards in Competitive Games: Inverse Game Theory with Entropy Regularization
Junyi Liao, Zihan Zhu, Ethan Fang +2
Estimating the unknown reward functions driving agents' behaviors is of central interest in inverse reinforcement learning and game theory. To tackle this problem, we develop a uni…
Rethinking Token Prediction: Tree-Structured Diffusion Language Model
Zihao Wu, Haoming Yang, Juncheng Dong +1
Discrete diffusion language models have emerged as a competitive alternative to auto-regressive language models, but training them efficiently under limited parameter and memory bu…
Boosting In-Context Learning in LLMs Through the Lens of Classical Supervised Learning
Korel Gundem, Juncheng Dong, Dennis Zhang +2
In-Context Learning (ICL) allows Large Language Models (LLMs) to adapt to new tasks with just a few examples, but their predictions often suffer from systematic biases, leading to…
Learning in Context, Guided by Choice: A Reward-Free Paradigm for Reinforcement Learning with Transformers
Juncheng Dong, Bowen He, Moyang Guo +3
In-context reinforcement learning (ICRL) leverages the in-context learning capabilities of transformer models (TMs) to efficiently generalize to unseen sequential decision-making t…
In-Context Reinforcement Learning From Suboptimal Historical Data
Juncheng Dong, Moyang Guo, Ethan X. Fang +2
Transformer models have achieved remarkable empirical successes, largely due to their in-context learning capabilities. Inspired by this, we explore training an autoregressive tran…