5 citations · 18 across the 9 of their papers we have counts for
9 papers
Constrained belief updates explain geometric structures in transformer representations
Mateusz Piotrowski, Paul M. Riechers, Daniel Filan +1
What computational structures emerge in transformers trained on next-token prediction? In this work, we provide evidence that transformers implement constrained Bayesian belief upd…
Quantifying Local Specialization in Deep Neural Networks
Shlomi Hod, Daniel Filan, Stephen Casper +2
A neural network is locally specialized to the extent that parts of its computational graph (i.e. structure) can be abstractly represented as performing some comprehensible sub-tas…
Clusterability in Neural Networks
Daniel Filan, Stephen Casper, Shlomi Hod +3
The learned weights of a neural network have often been considered devoid of scrutable internal structure. In this paper, however, we look for structure in the form of clusterabili…
Pruned Neural Networks are Surprisingly Modular
Daniel Filan, Shlomi Hod, Cody Wild +2
The learned weights of a neural network are often considered devoid of scrutable internal structure. To discern structure in these weights, we introduce a measurable notion of modu…
Exploring Hierarchy-Aware Inverse Reinforcement Learning
Chris Cundy, Daniel Filan
We introduce a new generative model for human planning under the Bayesian Inverse Reinforcement Learning (BIRL) framework which takes into account the fact that humans often plan u…
On the Impossibility of Supersized Machines
Ben Garfinkel, Miles Brundage, Daniel Filan +6
In recent years, a number of prominent computer scientists, along with academics in fields such as philosophy and physics, have lent credence to the notion that machines may one da…