335 citations · 526 across the 10 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CL2024
Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language
Jiayi Wang, Yao Lu, Maurice Weber +4
English, as a very high-resource language, enables the pretraining of high-quality large language models (LLMs). The same cannot be said for most other languages, as leading LLMs s…
cs.LG2024
Jet Expansions of Residual Computation
Yihong Chen, Xiangxiang Xu, Yao Lu +2
We introduce a framework for expanding residual computational graphs using jets, operators that generalize truncated Taylor series. Our method provides a systematic approach to dis…