1 citations · 1 across the 5 of their papers we have counts for
5 papers
Incoherence in Goal-Conditioned Autoregressive Models
Jacek Karwowski, Raymond Douglas
We investigate mathematically the notion of incoherence: a structural issue with reinforcement learning policies derived by naive goal-conditioning of autoregressive models. We foc…
Hilbert geometry of the symmetric positive-definite bicone: Application to the geometry of the extended Gaussian family
Jacek Karwowski, Frank Nielsen
The extended Gaussian family is the closure of the Gaussian family obtained by completing the Gaussian family with the counterpart elements induced by degenerate covariance or dege…
Characterizing stable regions in the residual stream of LLMs
Jett Janiak, Jacek Karwowski, Chatrik Singh Mangat +3
We identify stable regions in the residual stream of Transformers, where the model's output remains insensitive to small activation changes, but exhibits high sensitivity at region…
Limitations of Agents Simulated by Predictive Models
Raymond Douglas, Jacek Karwowski, Chan Bae +2
There is increasing focus on adapting predictive models into agent-like systems, most notably AI assistants based on language models. We outline two structural reasons for why thes…
Goodhart's Law in Reinforcement Learning
Jacek Karwowski, Oliver Hayman, Xingjian Bai +3
Implementing a reward function that perfectly captures a complex task in the real world is impractical. As a result, it is often appropriate to think of the reward function as a pr…