1 citations · 1 across the 5 of their papers we have counts for
5 papers
Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations
Yunao Zheng, Bin Wen, Xiaojie Wang
Transformers lack a native lookup mechanism, requiring repeated dense computation to recognize and reuse local static patterns. Lngram v1 introduces tokenizer-independent condition…
Lngram: N-gram Conditional Memory in Latent Space
Yunao Zheng, Guoyang Xia, Xiaojie Wang +1
Sequence modeling requires both compositional reasoning and local static knowledge retrieval, yet standard Transformers handle both through dense computation. Engram partially deco…
ROSA-Tuning: Enhancing Long-Context Modeling via Suffix Matching
Yunao Zheng, Xiaojie Wang, Lei Ren +1
Long-context capability and computational efficiency are among the central challenges facing today's large language models. Existing efficient attention methods reduce computationa…
Enhancing Complex Instruction Following for Large Language Models with Mixture-of-Contexts Fine-tuning
Yuheng Lu, ZiMeng Bai, Caixia Yuan +2
Large language models (LLMs) exhibit remarkable capabilities in handling natural language tasks; however, they may struggle to consistently follow complex instructions including th…
Phased Instruction Fine-Tuning for Large Language Models
Wei Pang, Chuan Zhou, Xiao-Hua Zhou +1
Instruction Fine-Tuning enhances pre-trained language models from basic next-word prediction to complex instruction-following. However, existing One-off Instruction Fine-Tuning (On…