41 citations · 93 across the 26 of their papers we have counts for
27 papers
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
DeepSeek-AI, :, Anyi Xu +585
The widespread adoption of long-horizon agents has made model workloads increasingly input-heavy. Although prior work has substantially reduced the cost of long-context computation…
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
Xin Cheng, Xingkai Yu, Chenze Shao +30
Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose lo…
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
DeepSeek-AI, Anyi Xu, Bangcai Lin +315
We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models -- DeepSeek-V4-Pro with 1.6T parameters (49B activated) and DeepSe…
General learned delegation by clones
Darren Li, Meiqi Chen, Chenze Shao +2
Frontier language models improve with additional test-time computation, but serial reasoning or uncoordinated parallel sampling can be compute-inefficient under fixed inference bud…
Continuous Autoregressive Language Models
Chenze Shao, Darren Li, Fandong Meng +1
The efficiency of large language models (LLMs) is fundamentally limited by their sequential, token-by-token generation process. We argue that overcoming this bottleneck requires a…
Continuous Visual Autoregressive Generation via Score Maximization
Chenze Shao, Fandong Meng, Jie Zhou
Conventional wisdom suggests that autoregressive models are used to process discrete data. When applied to continuous modalities such as visual data, Visual AutoRegressive modeling…
Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space
Zhengrui Ma, Yang Feng, Chenze Shao +3
We introduce SLED, an alternative approach to speech language modeling by encoding speech waveforms into sequences of continuous latent representations and modeling them autoregres…
Beyond Next Token Prediction: Patch-Level Training for Large Language Models
Chenze Shao, Fandong Meng, Jie Zhou
The prohibitive training costs of Large Language Models (LLMs) have emerged as a significant bottleneck in the development of next-generation LLMs. In this paper, we show that it i…
Understanding and Addressing the Under-Translation Problem from the Perspective of Decoding Objective
Chenze Shao, Fandong Meng, Jiali Zeng +1
Neural Machine Translation (NMT) has made remarkable progress over the past years. However, under-translation and over-translation remain two challenging problems in state-of-the-a…
Language Generation with Strictly Proper Scoring Rules
Chenze Shao, Fandong Meng, Yijin Liu +1
Language generation based on maximum likelihood estimation (MLE) has become the fundamental approach for text generation. Maximum likelihood estimation is typically performed by mi…
Non-autoregressive Machine Translation with Probabilistic Context-free Grammar
Shangtong Gui, Chenze Shao, Zhengrui Ma +3
Non-autoregressive Transformer(NAT) significantly accelerates the inference of neural machine translation. However, conventional NAT models suffer from limited expression power and…
Beyond MLE: Convex Learning for Text Generation
Chenze Shao, Zhengrui Ma, Min Zhang +1
Maximum likelihood estimation (MLE) is a statistical method used to estimate the parameters of a probability distribution that best explain the observed data. In the context of tex…
Non-autoregressive Streaming Transformer for Simultaneous Translation
Zhengrui Ma, Shaolei Zhang, Shoutao Guo +3
Simultaneous machine translation (SiMT) models are trained to strike a balance between latency and translation quality. However, training these models to achieve high quality while…
Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation
Zhengrui Ma, Chenze Shao, Shangtong Gui +2
Non-autoregressive translation (NAT) reduces the decoding latency but suffers from performance degradation due to the multi-modality problem. Recently, the structure of directed ac…
Rephrasing the Reference for Non-Autoregressive Machine Translation
Chenze Shao, Jinchao Zhang, Jie Zhou +1
Non-autoregressive neural machine translation (NAT) models suffer from the multi-modality problem that there may exist multiple possible translations of a source sentence, so the r…
Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation
Chenze Shao, Yang Feng
Non-autoregressive translation (NAT) models are typically trained with the cross-entropy loss, which forces the model outputs to be aligned verbatim with the target sentence and wi…
Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation
Chenze Shao, Zhengrui Ma, Yang Feng
Non-autoregressive models achieve significant decoding speedup in neural machine translation but lack the ability to capture sequential dependency. Directed Acyclic Transformer (DA…
One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation
Chenze Shao, Xuanfu Wu, Yang Feng
Non-autoregressive neural machine translation (NAT) suffers from the multi-modality problem: the source sentence may have multiple correct translations, but the loss function is ca…
Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation
Chenze Shao, Yang Feng
Neural networks tend to gradually forget the previously learned knowledge when learning multiple tasks sequentially from dynamic data distributions. This problem is called \textit{…
Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation
Yang Feng, Shuhao Gu, Dengji Guo +2
Although teacher forcing has become the main training paradigm for neural machine translation, it usually makes predictions only conditioned on past information, and hence lacks gl…
Sequence-Level Training for Non-Autoregressive Neural Machine Translation
Chenze Shao, Yang Feng, Jinchao Zhang +2
In recent years, Neural Machine Translation (NMT) has achieved notable results in various translation tasks. However, the word-by-word generation manner determined by the autoregre…
Modeling Coverage for Non-Autoregressive Neural Machine Translation
Yong Shan, Yang Feng, Chenze Shao
Non-Autoregressive Neural Machine Translation (NAT) has achieved significant inference speedup by generating all tokens simultaneously. Despite its high efficiency, NAT usually suf…
Generating Diverse Translation from Model Distribution with Dropout
Xuanfu Wu, Yang Feng, Chenze Shao
Despite the improvement of translation quality, neural machine translation (NMT) often suffers from the lack of diversity in its generation. In this paper, we propose to generate d…
Modeling Fluency and Faithfulness for Diverse Neural Machine Translation
Yang Feng, Wanying Xie, Shuhao Gu +4
Neural machine translation models usually adopt the teacher forcing strategy for training which requires the predicted sequence matches ground truth word by word and forces the pro…