From the 1 of 7 linked papers with an AI index.
4 papers · 1 filter
PRISM Edit: One Vector for All Temporal Answers
Chen Huang, Qi Zheng, Ruiqin Zheng +2
The paper proposes PRISM Edit, a method that updates large language models to handle changing temporal facts by learning a single representation that can be modulated for different…
Do Language Models Use Their Depth Efficiently?
Róbert Csordás, Christopher D. Manning, Christopher Potts
Modern LLMs are increasingly deep, and depth correlates with performance, albeit with diminishing returns. However, do these models use their depth efficiently? Do they compose mor…
MoEUT: Mixture-of-Experts Universal Transformers
Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber +2
Previous work on Universal Transformers (UTs) has demonstrated the importance of parameter sharing across layers. By allowing recurrence in depth, UTs have advantages over standard…
Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations
Róbert Csordás, Christopher Potts, Christopher D. Manning +1
The Linear Representation Hypothesis (LRH) states that neural networks learn to encode concepts as directions in activation space, and a strong version of the LRH states that model…