2 papers
cs.CL2025
Token-based Decision Criteria Are Suboptimal in In-context Learning
Hakaze Cho, Yoshihiro Sakai, Mariko Kato +3
In-Context Learning (ICL) typically utilizes classification criteria from output probabilities of manually selected label tokens. However, we argue that such token-based classifica…
cs.CL2024
Understanding Token Probability Encoding in Output Embeddings
Hakaze Cho, Yoshihiro Sakai, Kenshiro Tanaka +2
In this paper, we investigate the output token probability information in the output embedding of language models. We find an approximate common log-linear encoding of output token…