4 citations · 6 across the 4 of their papers we have counts for
5 papers
MassAlloc Attention: Let Attention Allocate Its Own Compute
Jingze Shi, Zhangyang Peng, Xianduo Li +6
FullAttn often assigns negligible normalized mass to much of the causal score space, yet dense kernels execute the complete post-score path after forming each QK tile. We introduce…
Trainable Dynamic Mask Sparse Attention
Jingze Shi, Yifan Wu, Yiran Peng +4
The increasing demand for long-context modeling in large language models (LLMs) is bottlenecked by the quadratic complexity of the standard self-attention mechanism. The community…
ReTok: Replacing Tokenizer to Enhance Representation Efficiency in Large Language Model
Shuhao Gu, Mengdi Zhao, Bowen Zhang +3
Tokenizer is an essential component for large language models (LLMs), and a tokenizer with a high compression rate can improve the model's representation and processing efficiency.…
Emu3: Next-Token Prediction is All You Need
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo +22
While next-token prediction is considered a promising path towards artificial general intelligence, it has struggled to excel in multimodal tasks, which are still dominated by diff…
CoIE: Chain-of-Instruct Editing for Multi-Attribute Face Manipulation
Zhenduo Zhang, Bo-Wen Zhang, Guang Liu
Current text-to-image editing models often encounter challenges with smoothly manipulating multiple attributes using a single instruction. Taking inspiration from the Chain-of-Thou…