687 citations · 1.7k across the 81 of their papers we have counts for
4 papers · 1 filter
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Wenhui Wang, Hangbo Bao, Shaohan Huang +2
We generalize deep self-attention distillation in MiniLM (Wang et al., 2020) by only using self-attention relation distillation for task-agnostic compression of pretrained Transfor…
Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Haozhe Ji, Pei Ke, Shaohan Huang +2
Commonsense explanation generation aims to empower the machine's sense-making capability by generating plausible explanations to statements against commonsense. While this task is…
Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Haozhe Ji, Pei Ke, Shaohan Huang +3
Despite the success of generative pre-trained language models on a series of text generation tasks, they still suffer in cases where reasoning over underlying commonsense knowledge…
DocBank: A Benchmark Dataset for Document Layout Analysis
Minghao Li, Yiheng Xu, Lei Cui +4
Document layout analysis usually relies on computer vision models to understand documents while ignoring textual information that is vital to capture. Meanwhile, high quality label…