1 citations · 1 across the 5 of their papers we have counts for
3 papers · 1 filter
Incoherence in Goal-Conditioned Autoregressive Models
Jacek Karwowski, Raymond Douglas
We investigate mathematically the notion of incoherence: a structural issue with reinforcement learning policies derived by naive goal-conditioning of autoregressive models. We foc…
Characterizing stable regions in the residual stream of LLMs
Jett Janiak, Jacek Karwowski, Chatrik Singh Mangat +3
We identify stable regions in the residual stream of Transformers, where the model's output remains insensitive to small activation changes, but exhibits high sensitivity at region…
Goodhart's Law in Reinforcement Learning
Jacek Karwowski, Oliver Hayman, Xingjian Bai +3
Implementing a reward function that perfectly captures a complex task in the real world is impractical. As a result, it is often appropriate to think of the reward function as a pr…