4 papers
Pre-training Universal Language Representation
Yian Li, Hai Zhao
Despite the well-developed cut-edge representation learning for language, most language representation models usually focus on specific levels of linguistic units. This work introd…
BURT: BERT-inspired Universal Representation from Learning Meaningful Segment
Yian Li, Hai Zhao
Although pre-trained contextualized language models such as BERT achieve significant performance on various downstream tasks, current language representation still only focuses on…
Learning Universal Representations from Word to Sentence
Yian Li, Hai Zhao
Despite the well-developed cut-edge representation learning for language, most language representation models usually focus on specific level of linguistic unit, which cause great…
BURT: BERT-inspired Universal Representation from Twin Structure
Yian Li, Hai Zhao
Pre-trained contextualized language models such as BERT have shown great effectiveness in a wide range of downstream Natural Language Processing (NLP) tasks. However, the effective…