Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers
Roberto Garcia, Jerry Liu, Ronny Junkins +3
Large language models (LLMs) store factual knowledge in their parameters. While recent work has shown that this knowledge resides in MLP layers, existing constructive and mechanist…
cs.LG2025
Constructing Efficient Fact-Storing MLPs for Transformers
Owen Dugan, Roberto Garcia, Ronny Junkins +5
The success of large language models (LLMs) can be attributed in part to their ability to efficiently store factual knowledge as key-value mappings within their MLP parameters. Rec…
cs.LG2025
Adaptive Rank Allocation: Speeding Up Modern Transformers with RaNA Adapters
Roberto Garcia, Jerry Liu, Daniel Sorvisto +1
Large Language Models (LLMs) are computationally intensive, particularly during inference. Neuron-adaptive techniques, which selectively activate neurons in Multi-Layer Perceptron…