154 citations · 273 across the 21 of their papers we have counts for
22 papers
Active Measuring in Reinforcement Learning With Delayed Negative Effects
Daiqi Gao, Ziping Xu, Aseel Rawashdeh +2
Measuring states in reinforcement learning (RL) can be costly in real-world settings and may negatively influence future outcomes. We introduce the Actively Observable Markov Decis…
Harnessing Causality in Reinforcement Learning With Bagged Decision Times
Daiqi Gao, Hsin-Yu Lai, Predrag Klasnja +1
We consider reinforcement learning (RL) for a class of problems with bagged decision times. A bag contains a finite sequence of consecutive decision times. The transition dynamics…
AI-Assisted Causal Pathway Diagram for Human-Centered Design
Ruican Zhong, Donghoon Shin, Rosemary Meza +3
This paper explores the integration of causal pathway diagrams (CPD) into human-centered design (HCD), investigating how these diagrams can enhance the early stages of the design p…
Effect-Invariant Mechanisms for Policy Generalization
Sorawit Saengkyongam, Niklas Pfister, Predrag Klasnja +2
Policy learning is an important component of many real-world learning systems. A major challenge in policy learning is how to adapt efficiently to unseen environments or tasks. Rec…
Assessing the Impact of Context Inference Error and Partial Observability on RL Methods for Just-In-Time Adaptive Interventions
Karine Karine, Predrag Klasnja, Susan A. Murphy +1
Just-in-Time Adaptive Interventions (JITAIs) are a class of personalized health interventions developed within the behavioral science community. JITAIs aim to provide the right typ…
Doubly robust nearest neighbors in factor models
Raaz Dwivedi, Caleb Chin, Sabina Tomkins +3
We introduce and analyze an improved variant of nearest neighbors (NN) for estimation with missing data in latent factor models. We consider a matrix completion problem with missin…
Counterfactual inference in sequential experiments
Raaz Dwivedi, Katherine Tian, Sabina Tomkins +3
We consider after-study statistical inference for sequentially designed experiments wherein multiple units are assigned treatments for multiple time points using treatment policies…
The Micro-Randomized Trial for Developing Digital Interventions: Experimental Design and Data Analysis Considerations
Tianchen Qian, Ashley E. Walton, Linda M. Collins +8
Just-in-time adaptive interventions (JITAIs) are time-varying adaptive interventions that use frequent opportunities for the intervention to be adapted--weekly, daily, or even many…
IntelligentPooling: Practical Thompson Sampling for mHealth
Sabina Tomkins, Peng Liao, Predrag Klasnja +1
In mobile health (mHealth) smart devices deliver behavioral treatments repeatedly over time to a user with the goal of helping the user adopt and maintain healthy behaviors. Reinfo…
Batch Policy Learning in Average Reward Markov Decision Processes
Peng Liao, Zhengling Qi, Runzhe Wan +2
We consider the batch (off-line) policy learning problem in the infinite horizon Markov Decision Process. Motivated by mobile health applications, we focus on learning a policy tha…
The Micro-Randomized Trial for Developing Digital Interventions: Experimental Design Considerations
Ashley E. Walton, Linda M. Collins, Predrag Klasnja +4
Just-in-time adaptive interventions (JITAIs) are time-varying adaptive interventions that use frequent opportunities for the intervention to be adapted such as weekly, daily, or ev…
The Micro-Randomized Trial for Developing Digital Interventions: Data Analysis Methods
Tianchen Qian, Michael A. Russell, Linda M. Collins +4
Although there is much excitement surrounding the use of mobile and wearable technology for the purposes of delivering interventions as people go through their day-to-day lives, da…
Translating Behavioral Theory into Technological Interventions: Case Study of an mHealth App to Increase Self-reporting of Substance-Use Related Data
Mashfiqui Rabbi, Meredith Philyaw-Kotov, Jinseok Li +11
Mobile health (mHealth) applications are a powerful medium for providing behavioral interventions, and systematic reviews suggest that theory-based interventions are more effective…
Rapidly Personalizing Mobile Health Treatment Policies with Limited Data
Sabina Tomkins, Peng Liao, Predrag Klasnja +2
In mobile health (mHealth), reinforcement learning algorithms that adapt to one's context without learning personalized policies might fail to distinguish between the needs of indi…
Off-Policy Estimation of Long-Term Average Outcomes with Applications to Mobile Health
Peng Liao, Predrag Klasnja, Susan Murphy
Due to the recent advancements in wearables and sensing technology, health scientists are increasingly developing mobile health (mHealth) interventions. In mHealth interventions, m…
Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity
Peng Liao, Kristjan Greenewald, Predrag Klasnja +1
With the recent evolution of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically deliver…
Estimating Time-Varying Causal Excursion Effect in Mobile Health with Binary Outcomes
Tianchen Qian, Hyesun Yoo, Predrag Klasnja +2
Advances in wearables and digital technology now make it possible to deliver behavioral mobile health interventions to individuals in their everyday life. The micro-randomized tria…
Linear mixed models with endogenous covariates: modeling sequential treatment effects with application to a mobile health study
Tianchen Qian, Predrag Klasnja, Susan A. Murphy
Mobile health is a rapidly developing field in which behavioral treatments are delivered to individuals via wearables or smartphones to facilitate health-related behavior change. M…
Practical Considerations for Data Collection and Management in Mobile Health Micro-randomized Trials
Nicholas J. Seewald, Shawna N. Smith, Andy Jinseok Lee +2
There is a growing interest in leveraging the prevalence of mobile technology to improve health by delivering momentary, contextualized interventions to individuals' smartphones. A…
Personalizing Intervention Probabilities By Pooling
Sabina Tomkins, Predrag Klasnja, Susan Murphy
In many mobile health interventions, treatments should only be delivered in a particular context, for example when a user is currently stressed, walking or sedentary. Even in an op…
Action Centered Contextual Bandits
Kristjan Greenewald, Ambuj Tewari, Predrag Klasnja +1
Contextual bandits have become popular as they offer a middle ground between very simple approaches based on multi-armed bandits and very complex approaches using the full power of…
Sample Size Calculations for Micro-randomized Trials in mHealth
Peng Liao, Predrag Klasnja, Ambuj Tewari +1
The use and development of mobile interventions are experiencing rapid growth. In "just-in-time" mobile interventions, treatments are provided via a mobile device and they are inte…