60 citations · 110 across the 12 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs
Woomin Song, Seunghyuk Oh, Sangwoo Mo +4
Large language models (LLMs) have shown remarkable performance in various natural language processing tasks. However, a primary constraint they face is the context limit, i.e., the…
cs.LG2020★ 30 cited
Regularizing Class-wise Predictions via Self-knowledge Distillation
Sukmin Yun, Jongjin Park, Kimin Lee +1
Deep neural networks with millions of parameters may suffer from poor generalization due to overfitting. To mitigate the issue, we propose a new regularization method that penalize…