1 citations · 1 across the 6 of their papers we have counts for
7 papers
Proxy Compression for Language Modeling
Lin Zheng, Xinyu Li, Qian Liu +2
Modern language models are trained almost exclusively on token sequences produced by a fixed tokenizer, an external lossless compressor often over UTF-8 byte sequences, thereby cou…
DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas
Zirui Wu, Lin Zheng, Zhihui Xie +8
Diffusion Language Models (DLMs) present a compelling alternative to autoregressive models, offering flexible, any-order infilling without specialized prompting design. However, th…
Dream 7B: Diffusion Large Language Models
Jiacheng Ye, Zhihui Xie, Lin Zheng +5
We introduce Dream 7B, the most powerful open diffusion large language model to date. Unlike autoregressive (AR) models that generate tokens sequentially, Dream 7B employs discrete…
Haste Makes Waste: Evaluating Planning Abilities of LLMs for Efficient and Feasible Multitasking with Time Constraints Between Actions
Zirui Wu, Xiao Liu, Jiayi Li +2
While Large Language Model-based agents have demonstrated substantial progress in task completion, existing evaluation benchmarks tend to overemphasize single-task performance, wit…
SubgoalXL: Subgoal-based Expert Learning for Theorem Proving
Xueliang Zhao, Lin Zheng, Haige Bo +3
Formal theorem proving, a field at the intersection of mathematics and computer science, has seen renewed interest with advancements in large language models (LLMs). This paper int…
Linear Attention via Orthogonal Memory
Jun Zhang, Shuyang Jiang, Jiangtao Feng +2
Efficient attentions have greatly improved the computational efficiency of Transformers. However, most existing linear attention mechanisms suffer from an \emph{efficiency degradat…
Self-Infilling Code Generation
Lin Zheng, Jianbo Yuan, Zhi Zhang +2
This work introduces self-infilling code generation, a general framework that incorporates infilling operations into auto-regressive decoding. Our approach capitalizes on the obser…