5 citations · 10 across the 16 of their papers we have counts for
Showing cs.CLShow all
3 papers · 1 filter
cs.CL2024
You Only Use Reactive Attention Slice For Long Context Retrieval
Yun Joon Soh, Hanxian Huang, Yuandong Tian +1
Supporting longer context for Large Language Models (LLM) is a promising direction to advance LLMs. As training a model for a longer context window is computationally expensive, ma…
cs.CL2024
Learning to Maximize Mutual Information for Chain-of-Thought Distillation
Xin Chen, Hanxian Huang, Yanjun Gao +3
Knowledge distillation, the technique of transferring knowledge from large, complex models to smaller ones, marks a pivotal step towards efficient AI deployment. Distilling Step-by…
cs.CL2021★ 1 cited
Learning Bounded Context-Free-Grammar via LSTM and the Transformer:Difference and Explanations
Hui Shi, Sicun Gao, Yuandong Tian +2
Long Short-Term Memory (LSTM) and Transformers are two popular neural architectures used for natural language processing tasks. Theoretical results show that both are Turing-comple…