8 citations · 14 across the 14 of their papers we have counts for
14 papers
Towards Understanding Specification Gaming in Reasoning Models
Kei Nishimura-Gasparian, Robert McCarthy, David Lindner
Specification gaming is a critical failure mode of LLM agents. Despite this, there has been little systematic research into when it arises and what drives it. To address this, we b…
Reasoning Models Struggle to Control their Chains of Thought
Chen Yueh-Han, Robert McCarthy, Bruce W. Lee +5
Chain-of-thought (CoT) monitoring is a promising tool for detecting misbehaviors and understanding the motivations of modern reasoning models. However, if models can control what t…
Split Personality Training: Revealing Latent Knowledge Through Alternate Personalities
Florian Dietz, William Wale, Oscar Gilg +5
Detecting misalignment in large language models is challenging because models may learn to conceal misbehavior during training. Standard auditing techniques fall short: black-box m…
Early Signs of Steganographic Capabilities in Frontier LLMs
Artur Zolkowski, Kei Nishimura-Gasparian, Robert McCarthy +2
Monitoring Large Language Model (LLM) outputs is crucial for mitigating risks from misuse and misalignment. However, LLMs could evade monitoring through steganography: Encoding hid…
Large language models can learn and generalize steganographic chain-of-thought under process supervision
Joey Skaf, Luis Ibanez-Lissen, Robert McCarthy +8
Chain-of-thought (CoT) reasoning not only enhances large language model performance but also provides critical insights into decision-making processes, marking it as a useful tool…
Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs
Yohan Mathew, Ollie Matthews, Robert McCarthy +4
The rapid proliferation of frontier model agents promises significant societal advances but also raises concerns about systemic risks arising from unsafe interactions. Collusion to…
Towards Generalist Robot Learning from Internet Video: A Survey
Robert McCarthy, Daniel C. H. Tan, Dominik Schmidt +5
Scaling deep learning to massive and diverse internet data has driven remarkable breakthroughs in domains such as video generation and natural language processing. Robot learning,…
Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World
Nico Gürtler, Felix Widmaier, Cansu Sancaktar +21
Experimentation on real robots is demanding in terms of time and costs. For this reason, a large part of the reinforcement learning (RL) community uses simulators to develop and be…
Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theory
Daniel C. H. Tan, Fernando Acero, Robert McCarthy +2
Guaranteeing safe behaviour of reinforcement learning (RL) policies poses significant challenges for safety-critical applications, despite RL's generality and scalability. To addre…
Identifying Expert Behavior in Offline Training Datasets Improves Behavioral Cloning of Robotic Manipulation Policies
Qiang Wang, Robert McCarthy, David Cordova Bulens +4
This paper presents our solution for the Real Robot Challenge (RRC) III, a competition featured in the NeurIPS 2022 Competition Track, aimed at addressing dexterous robotic manipul…
Improving Behavioural Cloning with Positive Unlabeled Learning
Qiang Wang, Robert McCarthy, David Cordova Bulens +6
Learning control policies offline from pre-recorded datasets is a promising avenue for solving challenging real-world problems. However, available datasets are typically of mixed q…
Imaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward Tasks
Robert McCarthy, Qiang Wang, Stephen J. Redmond
Model-based reinforcement learning is a promising learning strategy for practical robotic applications due to its improved data-efficiency versus model-free counterparts. However,…
Solving the Real Robot Challenge using Deep Reinforcement Learning
Robert McCarthy, Francisco Roldan Sanchez, Qiang Wang +4
This paper details our winning submission to Phase 1 of the 2021 Real Robot Challenge; a challenge in which a three-fingered robot must carry a cube along specified goal trajectori…
Real Robot Challenge: A Robotics Competition in the Cloud
Stefan Bauer, Felix Widmaier, Manuel Wüthrich +39
Dexterous manipulation remains an open problem in robotics. To coordinate efforts of the research community towards tackling this problem, we propose a shared benchmark. We designe…