Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
The Direction of Ignorance: LLMs Encode and Temper a Unigram Prior
Toni J. B. Liu, Jiajun Bao, Yizhou Liu +5
What does a language model predict when it has few clues?The answer lies in its unembedding geometry: a single direction of the unembedding matrix encodes the unigram distribution…
cs.LG2026
A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning
Jiajun Bao, Zihao Qi, Toni J. B. Liu +4
Pretrained large language models (LLMs) have demonstrated in-context learning (ICL) capabilities for numerical inference over sequences serialized as text. Prior work has identifie…