Showing 2025Show all
3 papers · 1 filter
cs.LG2025
MesaNet: Sequence Modeling by Locally Optimal Test-Time Training
Johannes von Oswald, Nino Scherrer, Seijin Kobayashi +14
Sequence modeling is currently dominated by causal transformer architectures that use softmax self-attention. Although widely adopted, transformers require scaling memory and compu…
cs.LG2025
Quartet: Native FP4 Training Can Be Optimal for Large Language Models
Roberto L. Castro, Andrei Panferov, Soroush Tabesh +5
Training large language models (LLMs) models directly in low-precision offers a way to address computational costs by improving both throughput and energy efficiency. For those pur…
cs.LG2025
DarwinLM: Evolutionary Structured Pruning of Large Language Models
Shengkun Tang, Oliver Sieberling, Eldar Kurtic +2
Large Language Models (LLMs) have achieved significant success across various NLP tasks. However, their massive computational costs limit their widespread use, particularly in real…