Language Model Pre-training for Hierarchical Document Representations

Ming-Wei Chang, Kristina Toutanova, Kenton Lee +1

Hierarchical neural architectures are often used to capture long-distance dependencies and have been applied to many document-level tasks such as summarization, document segmentati…