2.7k citations · 8.5k across the 41 of their papers we have counts for
22 papers · 1 filter
Deep State-Space Generative Model For Correlated Time-to-Event Predictions
Yuan Xue, Denny Zhou, Nan Du +4
Capturing the inter-dependencies among multiple types of clinically-critical events is critical not only to accurate future event prediction, but also to better treatment planning.…
Brainformers: Trading Simplicity for Efficiency
Yanqi Zhou, Nan Du, Yanping Huang +12
Transformers are central to recent successes in natural language processing and computer vision. Transformers have a mostly uniform backbone where layers alternate between feed-for…
Scaling Instruction-Finetuned Language Models
Hyung Won Chung, Le Hou, Shayne Longpre +32
Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we expl…
Mixture-of-Experts with Expert Choice Routing
Yanqi Zhou, Tao Lei, Hanxiao Liu +7
Sparsely-activated Mixture-of-experts (MoE) models allow the number of parameters to greatly increase while keeping the amount of computation for a given token or a given sample un…
BEDS-Bench: Behavior of EHR-models under Distributional Shift--A Benchmark
Anand Avati, Martin Seneviratne, Emily Xue +3
Machine learning has recently demonstrated impressive progress in predictive accuracy across a wide array of tasks. Most ML approaches focus on generalization performance on unseen…
MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records
Zhen Xu, David R. So, Andrew M. Dai
One important challenge of applying deep learning to electronic health records (EHR) is the complexity of their multimodal structure. EHR usually contains a mixture of structured (…
Training independent subnetworks for robust prediction
Marton Havasi, Rodolphe Jenatton, Stanislav Fort +5
Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over th…
Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss
Kamil Nar, Yuan Xue, Andrew M. Dai
When training the parameters of a linear dynamical model, the gradient descent algorithm is likely to fail to converge if the squared-error loss is used as the training loss functi…
Deep Physiological State Space Model for Clinical Forecasting
Yuan Xue, Denny Zhou, Nan Du +4
Clinical forecasting based on electronic medical records (EMR) can uncover the temporal correlations between patients' conditions and outcomes from sequences of longitudinal clinic…
Modelling EHR timeseries by restricting feature interaction
Kun Zhang, Yuan Xue, Gerardo Flores +3
Time series data are prevalent in electronic health records, mostly in the form of physiological parameters such as vital signs and lab tests. The patterns of these values may be s…
Federated and Differentially Private Learning for Electronic Health Records
Stephen R. Pfohl, Andrew M. Dai, Katherine Heller
The use of collaborative and decentralized machine learning techniques such as federated learning have the potential to enable the development and deployment of clinical risk predi…
Learning an Adaptive Learning Rate Schedule
Zhen Xu, Andrew M. Dai, Jonas Kemp +1
The learning rate is one of the most important hyper-parameters for model training and generalization. However, current hand-designed parametric learning rate schedules offer limit…
Improved Hierarchical Patient Classification with Language Model Pretraining over Clinical Notes
Jonas Kemp, Alvin Rajkomar, Andrew M. Dai
Clinical notes in electronic health records contain highly heterogeneous writing styles, including non-standard terminology or abbreviations. Using these notes in predictive modeli…
Explaining an increase in predicted risk for clinical alerts
Michaela Hardt, Alvin Rajkomar, Gerardo Flores +5
Much work aims to explain a model's prediction on a static input. We consider explanations in a temporal setting where a stateful dynamical model produces a sequence of risk estima…
Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer
Edward Choi, Zhen Xu, Yujia Li +4
Effective modeling of electronic health records (EHR) is rapidly becoming an important topic in both academia and industry. A recent study showed that using the graphical structure…
Analyzing the Role of Model Uncertainty for Electronic Health Records
Michael W. Dusenberry, Dustin Tran, Edward Choi +5
In medicine, both ethical and monetary costs of incorrect predictions can be significant, and the complexity of the problems often necessitates increasingly complex models. Recent…
Music Transformer
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit +7
Music relies heavily on repetition to build structure and meaning. Self-reference occurs on multiple timescales, from motifs to phrases to reusing of entire sections of music, such…
Learning Longer-term Dependencies in RNNs with Auxiliary Losses
Trieu H. Trinh, Andrew M. Dai, Minh-Thang Luong +1
Despite recent advances in training recurrent neural networks (RNNs), capturing long-term dependencies in sequences remains a fundamental challenge. Most approaches use backpropaga…
Who Said What: Modeling Individual Labelers Improves Classification
Melody Y. Guan, Varun Gulshan, Andrew M. Dai +1
Data are often labeled by many different experts with each expert only labeling a small fraction of the data and each data point being labeled by several experts. This reduces the…
HyperNetworks
David Ha, Andrew Dai, Quoc V. Le
This work explores hypernetworks: an approach of using a one network, also known as a hypernetwork, to generate the weights for another network. Hypernetworks provide an abstractio…
Semi-supervised Sequence Learning
Andrew M. Dai, Quoc V. Le
We present two approaches that use unlabeled data to improve sequence learning with recurrent networks. The first approach is to predict what comes next in a sequence, which is a c…
Generating Sentences from a Continuous Space
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals +3
The standard recurrent neural network language model (RNNLM) generates sentences one word at a time and does not work from an explicit global sentence representation. In this work,…