11 citations · 12 across the 4 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CL2024
Fewer Truncations Improve Language Modeling
Hantian Ding, Zijian Wang, Giovanni Paolini +4
In large language model training, input documents are typically concatenated together and then split into sequences of equal length to avoid padding tokens. Despite its efficiency,…
cs.CL2024★ 1 cited
Code Representation Learning At Scale
Dejiao Zhang, Wasi Ahmad, Ming Tan +5
Recent studies have shown that code language models at scale demonstrate significant performance gains on downstream tasks, i.e., code generation. However, most of the existing wor…