29 citations · 43 across the 13 of their papers we have counts for
5 papers · 1 filter
Learning Code Preference via Synthetic Evolution
Jiawei Liu, Thanh Nguyen, Mingyue Shang +5
Large Language Models (LLMs) have recently demonstrated remarkable coding capabilities. However, assessing code generation based on well-formed properties and aligning it with deve…
Planning-Aware Code Infilling via Horizon-Length Prediction
Yifeng Ding, Hantian Ding, Shiqi Wang +3
Fill-in-the-Middle (FIM), or infilling, has become integral to code language models, enabling generation of missing code given both left and right contexts. However, the current FI…
Fewer Truncations Improve Language Modeling
Hantian Ding, Zijian Wang, Giovanni Paolini +4
In large language model training, input documents are typically concatenated together and then split into sequences of equal length to avoid padding tokens. Despite its efficiency,…
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs
Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda +10
This study introduces bifurcated attention, a method designed to enhance language model inference in shared-context batch decoding scenarios. Our approach addresses the challenge o…
Code Representation Learning At Scale
Dejiao Zhang, Wasi Ahmad, Ming Tan +5
Recent studies have shown that code language models at scale demonstrate significant performance gains on downstream tasks, i.e., code generation. However, most of the existing wor…