106 citations · 234 across the 32 of their papers we have counts for
4 papers · 1 filter
Learning to Read Out: Unembedding Dynamics in Language Model Pretraining
Matteo He, William F. Shen, Alex Iacob +3
When does a language model acquire a capability? Developmental studies usually answer this by tracking when models start producing the correct token. But that prediction passes thr…
Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens
Matteo He, William F. Shen, Xinchi Qiu +1
A language model's prediction of its next token develops across layers, and lens methods track this process by decoding intermediate hidden states into tokens. But a lens reading r…
Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learning
Wannan, Yang, Xinchi Qiu +6
Large Language Models (LLMs) exhibit impressive capabilities but often hallucinate, confidently providing incorrect answers instead of admitting ignorance. Prior work has shown tha…
Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages
Wanru Zhao, Yihong Chen, Royson Lee +4
Pre-trained large language models (LLMs) have become a cornerstone of modern natural language processing, with their capabilities extending across a wide range of applications and…