7 papers · 1 filter
Metonymic Circuits for Abstract Concept Grounding in Vision Transformers
Jing Ding, Ziqiao Ma, Jiayuan Mao +2
We study how Vision Transformers ground abstract concepts (e.g., angry) when training data provide limited direct referential evidence. We hypothesize a metonymic grounding mechani…
Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability
Yu Mao, Lei Yu, Zining Zhu +6
We connect the spurious-reward paradox to a model's reachability and propose random-reward reinforcement learning (RL) as a useful tool for the probing enterprise, addressing a dec…
Cross-Session Decomposition Attacks: Scaling Risk and Intent-Aligned Retrieval Defense
Disen Liao, Yihan Wang, Freda Shi +1
Scaling laws are usually read as a capability story: lower language-modeling loss yields more useful models. We study a safety consequence of this mechanism in \emph{cross-session…
LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering
Xuye Liu, Yimu Wang, Peng Shi +7
Scientific literature is increasingly used as a knowledge source for language models, retrieval-augmented generation systems, and research assistants, but answering research questi…
A Language Model from 1913: Pretraining on Historical Text
Xiaoxi Luo, Zachary Shinnick, Niclas Griesshaber +5
While modern language models increasingly rely on ever-larger web corpora, we show that pretraining on historical text (e.g., pre-1913 text) in a data-constrained setting can produ…
How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Them
Disen Liao, Freda Shi
Tokenization is the first step in every language model (LM), yet it never takes the sounds of words into account. We investigate how tokenization influences text-only LMs' ability…
Translation or Recitation? Calibrating Evaluation Scores for Machine Translation of Extremely Low-Resource Languages
Danlu Chen, Ka Sing He, Jiahe Tian +4
The landscape of extremely low-resource machine translation (MT) is characterized by perplexing variability in reported performance, often making results across different language…