3 papers
cs.LG2026
MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers
Roberto Garcia, Jerry Liu, Ronny Junkins +3
Large language models (LLMs) store factual knowledge in their parameters. While recent work has shown that this knowledge resides in MLP layers, existing constructive and mechanist…
cs.LG2024
Restructuring Vector Quantization with the Rotation Trick
Christopher Fifty, Ronald G. Junkins, Dennis Duan +5
Vector Quantized Variational AutoEncoders (VQ-VAEs) are designed to compress a continuous input to a discrete latent space and reconstruct it with minimal distortion. They operate…
cs.LG2023
Context-Aware Meta-Learning
Christopher Fifty, Dennis Duan, Ronald G. Junkins +4
Large Language Models like ChatGPT demonstrate a remarkable capacity to learn new concepts during inference without any fine-tuning. However, visual models trained to detect new ob…