37 citations · 47 across the 4 of their papers we have counts for
4 papers
Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters
Yang Xiang, Zhihua Wu, Weibao Gong +15
The ever-growing model size and scale of compute have attracted increasing interests in training deep learning models over multiple nodes. However, when it comes to training on clo…
Multimodal data matters: language model pre-training over structured and unstructured electronic health records
Sicen Liu, Xiaolong Wang, Yongshuai Hou +5
As two important textual modalities in electronic health records (EHR), both structured data (clinical codes) and unstructured data (clinical narratives) have recently been increas…
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
Shuohuan Wang, Yu Sun, Yang Xiang +26
Pre-trained language models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. GPT-3 has shown that scaling up pre-trained language models c…
Machine Learning for Multimodal Electronic Health Records-based Research: Challenges and Perspectives
Ziyi Liu, Jiaqi Zhang, Yongshuai Hou +3
Background: Electronic Health Records (EHRs) contain rich information of patients' health history, which usually include both structured and unstructured data. There have been many…