Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Masked Gated Linear Unit
Yukito Tajima, Nakamasa Inoue, Yusuke Sekikawa +2
Gated Linear Units (GLUs) have become essential components in the feed-forward networks of state-of-the-art Large Language Models (LLMs). However, they require twice as many memory…
cs.LG2025
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
Kazuki Fujii, Yukito Tajima, Sakae Mizuki +14
The performance of large language models (LLMs) in program synthesis and mathematical reasoning is fundamentally limited by the quality of their pre-training corpora. We introduce…