Publications (16)
Understanding In-context Learning of Addition via Activation Subspaces
Xinyan Hu, Kayo Yin, Michael I. Jordan +2
To perform few-shot learning, language models extract signals from a few input-label pairs, aggregate these into a learned prediction rule, and apply this rule to new inputs. How i…
American Sign Language Handshapes Reflect Pressures for Communicative Efficiency
Kayo Yin, Terry Regier, Dan Klein
Communicative efficiency is a key topic in linguistics and cognitive psychology, with many studies demonstrating how the pressure to communicate with minimal effort guides the form…
When Does Translation Require Context? A Data-driven, Multilingual Exploration
Patrick Fernandes, Kayo Yin, Emmy Liu +2
Although proper handling of discourse significantly contributes to the quality of machine translation (MT), these improvements are not adequately measured in common translation qua…
Including Signed Languages in Natural Language Processing
Kayo Yin, Amit Moryossef, Julie Hochgesang +2
Signed languages are the primary means of communication for many deaf and hard of hearing individuals. Since signed languages exhibit all the fundamental linguistic properties of n…
Better Sign Language Translation with STMC-Transformer
Kayo Yin, Jesse Read
Sign Language Translation (SLT) first uses a Sign Language Recognition (SLR) system to extract sign language glosses from videos. Then, a translation system generates spoken langua…
Which Attention Heads Matter for In-Context Learning?
Kayo Yin, Jacob Steinhardt
Large language models (LLMs) exhibit impressive in-context learning (ICL) capability, enabling them to perform new tasks using only a few demonstrations in the prompt. Two differen…