41 citations · 135 across the 19 of their papers we have counts for
21 papers
TimeCap: A Scalable Open-Source Pipeline for Captioning Wearable Health Time-Series Data
Max Rosenblattl, Milana Gurbanova, Karl Aurel Deck +16
Time-series data generated continuously across industrial, environmental, and especially health domains produce multimodal signals whose scale and heterogeneity make their processi…
Uncertainty-Aware Selection of Online Algorithms with Simulator Ensembles
Yongyi Guo, Zifan Xu, Ziping Xu +1
The performance of online reinforcement learning depends critically on design choices, especially those that affect exploration. These choices are often selected by fitting a simul…
OpenMHC: Accelerating the Science of Wearable Foundation Models
Narayan Schuetz, Yuze Bai, Lianggang Pan +16
Mobile and wearable devices offer an unprecedented opportunity for continuous, passive health monitoring and active health coaching. However, the largest wearable datasets are not…
Tabular Foundation Models Can Do Survival Analysis
Da In Kim, Wei Siang Lai, Kelly W. Zhang
While tabular foundation models have achieved remarkable success in classification and regression, adapting them to model time-to-event outcomes for survival analysis is non-trivia…
Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions
Asim H. Gazi, Yongyi Guo, Daiqi Gao +3
Reinforcement learning (RL) has achieved remarkable success in real-world decision-making across diverse domains, including gaming, robotics, online advertising, public health, and…
Reproducible workflow for online AI in digital health
Susobhan Ghosh, Bhanu T. Gullapalli, Daiqi Gao +5
Online artificial intelligence (AI) algorithms are an important component of digital health interventions. These online algorithms are designed to continually learn and improve the…
A Deployed Online Reinforcement Learning Algorithm In An Oral Health Clinical Trial
Anna L. Trella, Kelly W. Zhang, Hinal Jajal +4
Dental disease is a prevalent chronic condition associated with substantial financial burden, personal suffering, and increased risk of systemic diseases. Despite widespread recomm…
Effective Monitoring of Online Decision-Making Algorithms in Digital Intervention Implementation
Anna L. Trella, Susobhan Ghosh, Erin E. Bonar +10
Online AI decision-making algorithms are increasingly used by digital interventions to dynamically personalize treatment to individuals. These algorithms determine, in real-time, t…
Replicable Bandits for Digital Health Interventions
Kelly W. Zhang, Nowell Closser, Anna L. Trella +1
Adaptive treatment assignment algorithms, such as bandit algorithms, are increasingly used in digital health intervention clinical trials. Frequently, the data collected from these…
Oralytics Reinforcement Learning Algorithm
Anna L. Trella, Kelly W. Zhang, Stephanie M. Carpenter +6
Dental disease is still one of the most common chronic diseases in the United States. While dental disease is preventable through healthy oral self-care behaviors (OSCB), this basi…
The Fallacy of Minimizing Cumulative Regret in the Sequential Task Setting
Ziping Xu, Kelly W. Zhang, Susan A. Murphy
Online Reinforcement Learning (RL) is typically framed as the process of minimizing cumulative regret (CR) through interactions with an unknown environment. However, real-world RL…
Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani +4
Online reinforcement learning (RL) algorithms offer great potential for personalizing treatment for participants in clinical trials. However, deploying an online, autonomous algori…
Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling
Susobhan Ghosh, Raphael Kim, Prasidh Chhabria +5
There is a growing interest in using reinforcement learning (RL) to personalize sequences of treatments in digital health to support users in adopting healthier behaviors. Such seq…
Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani +3
Dental disease is one of the most common chronic diseases despite being largely preventable. However, professional advice on optimal oral hygiene practices is often forgotten or ab…
A Bayesian Approach to Learning Bandit Structure in Markov Decision Processes
Kelly W. Zhang, Omer Gottesman, Finale Doshi-Velez
In the reinforcement learning literature, there are many algorithms developed for either Contextual Bandit (CB) or Markov Decision Processes (MDP) environments. However, when deplo…
Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani +3
Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in d…
Statistical Inference After Adaptive Sampling for Longitudinal Data
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
Online reinforcement learning and other adaptive sampling algorithms are increasingly used in digital intervention experiments to optimize treatment delivery for users over time. I…
Statistical Inference with M-Estimators on Adaptively Collected Data
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
Bandit algorithms are increasingly used in real-world sequential decision-making problems. Associated with this is an increased desire to be able to use the resulting datasets to a…
Inference for Batched Bandits
Kelly W. Zhang, Lucas Janson, Susan A. Murphy
As bandit algorithms are increasingly utilized in scientific studies and industrial applications, there is an associated increasing need for reliable inference methods based on the…
Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis
Kelly W. Zhang, Samuel R. Bowman
Recent work using auxiliary prediction task classifiers to investigate the properties of LSTM representations has begun to shed light on why pretrained representations, like ELMo (…
Adversarially Regularized Autoencoders
Jake Zhao, Yoon Kim, Kelly Zhang +2
Deep latent variable models, trained using variational autoencoders or generative adversarial networks, are now a key technique for representation learning of continuous structures…