6 papers · 1 filter
When Cosine Similarity Fails to Reflect Linearly Accessible Structure in Dialogue Models
Yu Sun, Mengyin Lu, Cong Feng +2
Cosine similarity is widely used to analyze transformer representations, implicitly assuming that similarity reflects task-relevant structure. We study when this assumption fails i…
K/V-Cache Interventions Dissociate Representation Alignment from Persona Expression in Decoder-Only Language Models
Yu Sun, Mengyin Lu, Cong Feng +2
We study K/V-cache interventions -- transplanting a target-conditioned K/V trajectory into a source-persona generation -- as a structured surface for persona control in decoder-onl…
OpenCompass: A Universal Evaluation Platform for Large Language Models
Maosong Cao, Kai Chen, Haodong Duan +27
In recent years, the field of artificial intelligence has undergone a paradigm shift from task-specific small-scale models to general-purpose large language models (LLMs). With the…
SPEAR: Code-Augmented Agentic Prompt Optimization
Mengyin Lu, Cong Feng, Huimin Han +6
Automatic prompt engineering (APE) rewrites prompts to improve downstream task performance, but existing APE loops treat the optimizer itself as a fixed pipeline. We port the code-…
Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping
Yao Chen, Yilong Chen, Yinqi Yang +9
Existing approaches to increasing the effective depth of Transformers predominantly rely on parameter reuse, extending computation through recursive execution. Under this paradigm,…
ATTNPO: Attention-Guided Process Supervision for Efficient Reasoning
Shuaiyi Nie, Siyu Ding, Wenyuan Zhang +7
Large reasoning models trained with reinforcement learning and verifiable rewards (RLVR) achieve strong performance on complex reasoning tasks, yet often overthink, generating redu…