From the 1 of 14 linked papers with an AI index.
14 papers
Data-Efficient Adaptation of LLMs via Attention Head Reweighting
Tuomas Oikarinen, Zixiao Chen, Charlotte Siska +3
The paper introduces Attention Head Reweighting (AHR), a method that adapts large language models to new text‑classification tasks by learning a single scalar weight per attention…
Generative causal testing to bridge data-driven models and scientific theories in language neuroscience
Richard Antonello, Chandan Singh, Shailee Jain +5
Representations from large language models are highly effective at predicting BOLD fMRI responses to language stimuli. However, these representations are largely opaque: it is uncl…
Test-time Recursive Thinking: Self-Improvement without External Feedback
Yufan Zhuang, Chandan Singh, Liyuan Liu +5
Modern Large Language Models (LLMs) have shown rapid improvements in reasoning capabilities, driven largely by reinforcement learning (RL) with verifiable rewards. Here, we ask whe…
Interpretable Next-token Prediction via the Generalized Induction Head
Eunji Kim, Sriya Mantena, Weiwei Yang +3
While large transformer models excel in predictive performance, their lack of interpretability restricts their usefulness in high-stakes domains. To remedy this, we propose the Gen…
Text Generation Beyond Discrete Token Sampling
Yufan Zhuang, Liyuan Liu, Chandan Singh +2
In standard autoregressive generation, an LLM predicts the next-token distribution, samples a discrete token, and then discards the distribution, passing only the sampled token as…
Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
Zeliang Zhang, Xiaodong Liu, Hao Cheng +2
By increasing model parameters but activating them sparsely when performing a task, the use of Mixture-of-Experts (MoE) architecture significantly improves the performance of Large…