144 citations · 611 across the 31 of their papers we have counts for
10 papers · 1 filter
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations
Peng Jin, Jinfa Huang, Fenglin Liu +5
Most video-and-language representation learning approaches employ contrastive learning, e.g., CLIP, to project the video and text features into a common latent space according to t…
Retrieval-Augmented and Knowledge-Grounded Language Models for Faithful Clinical Medicine
Fenglin Liu, Bang Yang, Chenyu You +6
Language models (LMs), including large language models (such as ChatGPT), have the potential to assist clinicians in generating various clinical notes. However, LMs are prone to pr…
MiniALBERT: Model Distillation via Parameter-Efficient Recursive Transformers
Mohammadmahdi Nouriborji, Omid Rohanian, Samaneh Kouchaki +1
Pre-trained Language Models (LMs) have become an integral part of Natural Language Processing (NLP) in recent years, due to their superior performance in downstream applications. I…
On the Effectiveness of Compact Biomedical Transformers
Omid Rohanian, Mohammadmahdi Nouriborji, Samaneh Kouchaki +1
Language models pre-trained on biomedical corpora, such as BioBERT, have recently shown promising results on downstream biomedical tasks. Many existing pre-trained models, on the o…
COPER: Continuous Patient State Perceiver
Vinod Kumar Chauhan, Anshul Thakur, Odhran O'Donoghue +1
In electronic health records (EHRs), irregular time-series (ITS) occur naturally due to patient health dynamics, reflected by irregular hospital visits, diseases/conditions and the…
Mixture of Input-Output Hidden Markov Models for Heterogeneous Disease Progression Modeling
Taha Ceritli, Andrew P. Creagh, David A. Clifton
A particular challenge for disease progression modeling is the heterogeneity of a disease and its manifestations in the patients. Existing approaches often assume the presence of a…
ProSelfLC: Progressive Self Label Correction Towards A Low-Temperature Entropy State
Xinshao Wang, Yang Hua, Elyor Kodirov +3
There is a family of label modification approaches including self and non-self label correction (LC), and output regularisation. They are widely used for training robust deep neura…
Multimodal Learning with Transformers: A Survey
Peng Xu, Xiatian Zhu, David A. Clifton
Transformer is a promising neural network learner, and has achieved great success in various machine learning tasks. Thanks to the recent prevalence of multimodal applications and…
Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data
Hang Yuan, Shing Chan, Andrew P. Creagh +4
Advances in deep learning for human activity recognition have been relatively limited due to the lack of large labelled datasets. In this study, we leverage self-supervised learnin…
How to Understand Masked Autoencoders
Shuhao Cao, Peng Xu, David A. Clifton
"Masked Autoencoders (MAE) Are Scalable Vision Learners" revolutionizes the self-supervised learning method in that it not only achieves the state-of-the-art for image pre-training…