Emergent Linear Representations in World Models of Self-Supervised Sequence Models

Neel Nanda, Andrew Lee, Martin Wattenberg

How do sequence models represent their decision-making process? Prior work suggests that Othello-playing neural network learned nonlinear models of the board state (Li et al., 2023…