Future Lens: Anticipating Subsequent Tokens from a Single Hidden State

Abstract

We conjecture that hidden state vectors corresponding to individual input tokens encode information sufficient to accurately predict several tokens ahead. More concretely, in this paper we ask: Given a hidden (internal) representation of a single token at position $t$ in an input, can we reliably anticipate the tokens that will appear at positions $\ge t+2$? To test this, we measure linear approximation and causal intervention methods in GPT-J-6B to evaluate the degree to which individual hidden states in the network contain signal rich enough to predict future hidden states and, ultimately, token outputs. We find that, at some layers, we can approximate a model's output with more than 48% accuracy with respect to its prediction of subsequent tokens through a single hidden state. Finally we present a "Future Lens" visualization that uses these methods to create a new view of transformer states.

arXiv commentAccepted at CoNLL 2023

References in corpus (13)

• Emergent Abilities of Large Language Models
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Locating and Editing Factual Associations in GPT
• What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
• Mass-Editing Memory in a Transformer
• Confident Adaptive Language Modeling
• Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping
• Eliciting Latent Predictions from Transformers with the Tuned Lens
• Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
• Finding Neurons in a Haystack: Case Studies with Sparse Probing
• Evaluating the Zero-shot Robustness of Instruction-tuned Language Models
• Jump to Conclusions: Short-Cutting Transformers With Linear Transformations