439 citations · 503 across the 16 of their papers we have counts for
17 papers
Informational Antilocality and the Locality Bias in LLMs
Andrew McInnerney, Shane Storks, Steven Abney +1
We consider the ability of transformer-based language models (LLMs) to learn what we call k-antilocal languages, i.e., languages that have no mutual information across any span of…
Conflict and Congruency Effects in Large Language Models: In-Weight and In-Context Competition in a Verbal Conflict Task
Xiaoyang Hu, Mike Angstadt, Shane Storks +5
Congruency effects, observed in conflict tasks such as Stroop and flanker tasks, have been investigated for nearly a century in psychology and neuroscience, but their mechanistic b…
Cognitive Convergence: Deep Similarities Between Large Language Models and Human Cognition
Chandra Sripada, Richard Lewis
LLMs are widely regarded as alien intelligences, systems whose cognitive operations are fundamentally unlike our own. Apparent similarities to human cognition are therefore often s…
In-Context Analogical Reasoning with Pre-Trained Language Models
Xiaoyang Hu, Shane Storks, Richard L. Lewis +1
Analogical reasoning is a fundamental capacity of human cognition that allows us to reason abstractly about novel situations by relating them to past experiences. While it is thoug…
Composing Task Knowledge with Modular Successor Feature Approximators
Wilka Carvalho, Angelos Filos, Richard L. Lewis +2
Recently, the Successor Features and Generalized Policy Improvement (SF&GPI) framework has been proposed as a method for learning, composing, and transferring predictive knowledge…
Large Language Models can Implement Policy Iteration
Ethan Brooks, Logan Walls, Richard L. Lewis +1
This work presents In-Context Policy Iteration, an algorithm for performing Reinforcement Learning (RL), in-context, using foundation models. While the application of foundation mo…
GrASP: Gradient-Based Affordance Selection for Planning
Vivek Veeriah, Zeyu Zheng, Richard Lewis +1
Planning with a learned model is arguably a key component of intelligence. There are several challenges in realizing such a component in large-scale reinforcement learning (RL) pro…
Accounting for Agreement Phenomena in Sentence Comprehension with Transformer Language Models: Effects of Similarity-based Interference on Surprisal and Attention
Soo Hyun Ryu, Richard L. Lewis
We advance a novel explanation of similarity-based interference effects in subject-verb and reflexive pronoun agreement processing, grounded in surprisal values computed from a pre…
Reinforcement Learning of Implicit and Explicit Control Flow in Instructions
Ethan A. Brooks, Janarthanan Rajendran, Richard L. Lewis +1
Learning to flexibly follow task instructions in dynamic environments poses interesting challenges for reinforcement learning agents. We focus here on the problem of learning contr…
Adaptive Pairwise Weights for Temporal Credit Assignment
Zeyu Zheng, Risto Vuorio, Richard Lewis +1
How much credit (or blame) should an action taken in a state get for a future reward? This is the fundamental temporal credit assignment problem in Reinforcement Learning (RL). One…
Learning State Representations from Random Deep Action-conditional Predictions
Zeyu Zheng, Vivek Veeriah, Risto Vuorio +2
Our main contribution in this work is an empirical finding that random General Value Functions (GVFs), i.e., deep action-conditional predictions -- random both in what feature of o…
Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment
Wilka Carvalho, Anthony Liang, Kimin Lee +4
First-person object-interaction tasks in high-fidelity, 3D, simulated environments such as the AI2Thor virtual home-environment pose significant sample-efficiency challenges for re…
How Should an Agent Practice?
Janarthanan Rajendran, Richard Lewis, Vivek Veeriah +2
We present a method for learning intrinsic reward functions to drive the learning of an agent during periods of practice in which extrinsic task rewards are not available. During p…
Discovery of Useful Questions as Auxiliary Tasks
Vivek Veeriah, Matteo Hessel, Zhongwen Xu +6
Arguably, intelligent agents ought to be able to discover their own questions so that in learning answers for them they learn unanticipated useful knowledge and skills; this depart…
Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games
Xiaoxiao Guo, Satinder Singh, Richard Lewis +1
Monte Carlo Tree Search (MCTS) methods have proven powerful in planning for sequential decision-making problems such as Go and video games, but their performance can be poor when t…
Action-Conditional Video Prediction using Deep Networks in Atari Games
Junhyuk Oh, Xiaoxiao Guo, Honglak Lee +2
Motivated by vision-based reinforcement learning (RL) problems, in particular Atari games from the recent benchmark Aracade Learning Environment (ALE), we consider spatio-temporal…
Variance-Based Rewards for Approximate Bayesian Reinforcement Learning
Jonathan Sorg, Satinder Singh, Richard L. Lewis
The explore{exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a…