Federated pretraining and fine tuning of BERT using clinical notes from multiple silos

Dianbo Liu, Tim Miller

Large scale contextual representation models, such as BERT, have significantly advanced natural language processing (NLP) in recently years. However, in certain area like healthcar…