90 citations · 120 across the 9 of their papers we have counts for
9 papers
On the generalization of language models from in-context learning and finetuning: a controlled study
Andrew K. Lampinen, Arslan Chaudhry, Stephanie C. Y. Chan +7
Large language models exhibit exciting capabilities, yet can show surprisingly narrow generalization from finetuning. E.g. they can fail to generalize to simple reversals of relati…
Uncovering Layer-Dependent Activation Sparsity Patterns in ReLU Transformers
Cody Wild, Jesper Anderson
Previous work has demonstrated that MLPs within ReLU Transformers exhibit high levels of sparsity, with many of their activations equal to zero for any given token. We build on tha…
An Empirical Investigation of Representation Learning for Imitation
Xin Chen, Sam Toyer, Cody Wild +9
Imitation learning often needs a large demonstration set in order to handle the full range of situations that an agent might find itself in during deployment. However, collecting e…
The MineRL BASALT Competition on Learning from Human Feedback
Rohin Shah, Cody Wild, Steven H. Wang +10
The last decade has seen a significant increase of interest in deep learning research, with many public successes that have demonstrated its potential. As such, these systems are n…
Clusterability in Neural Networks
Daniel Filan, Stephen Casper, Shlomi Hod +3
The learned weights of a neural network have often been considered devoid of scrutable internal structure. In this paper, however, we look for structure in the form of clusterabili…
Pruned Neural Networks are Surprisingly Modular
Daniel Filan, Shlomi Hod, Cody Wild +2
The learned weights of a neural network are often considered devoid of scrutable internal structure. To discern structure in these weights, we introduce a measurable notion of modu…
Adversarial Policies: Attacking Deep Reinforcement Learning
Adam Gleave, Michael Dennis, Cody Wild +3
Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers. However, a…
ALOHA: Auxiliary Loss Optimization for Hypothesis Augmentation
Ethan M. Rudd, Felipe N. Ducau, Cody Wild +2
Malware detection is a popular application of Machine Learning for Information Security (ML-Sec), in which an ML classifier is trained to predict whether a given file is malware or…
A Deep Learning Approach to Fast, Format-Agnostic Detection of Malicious Web Content
Joshua Saxe, Richard Harang, Cody Wild +1
Malicious web content is a serious problem on the Internet today. In this paper we propose a deep learning approach to detecting malevolent web pages. While past work on web conten…