AI + Digital Health Intervention Trial Design
Adaptive Micro-randomized Trials for Digital Health Interventions using Reinforcement Learning (RL)
Bibhas Chakraborty, Centre for Quantitative Medicine, Duke-NUS, Singapore
Adrian Aguilera, University of California, Berkeley, USA
Courtney Lyles, University of California, Davis, USA
Joseph Jay Williams, University of Toronto, Canada
Caroline Figueroa, TU Delft, The Netherlands
Nina Deliu, Sapienza University of Rome, Italy
Just-in-time adaptive interventions (JITAIs) [1] represent a class of behavioural interventions that aim to deliver the most appropriate support (e.g., push or nudge to promote healthy behaviours or warn against unhealthy ones) at the right occasion, considering an individual’s changing status and contexts. Against a backdrop of enhanced societal interest in population health, JITAIs are becoming increasingly popular in tandem with advances in mobile and wearable sensor technologies. The micro-randomized trial (MRT) design [1-4] is an innovative trial design that can aid the construction of optimized data-driven JITAIs and involves sequential, within-person randomization over many instances. The basic MRT design can be further improved to make it adaptive, thereby enabling it to learn ‘online’ from accumulated data as the trial progresses. This is appealing from an ethical perspective since the adaptive learning tends to make better interventions available to the trial participants. Adaptive learning is often operationalized via Reinforcement Learning (RL), in particular, contextual multi-arm bandit algorithms, e.g. Thompson Sampling. The goal of our research is to develop cutting-edge and statistically rigorous RL algorithms for MRTs or for general deployment of JITAIs, tailored to specific health problems, thereby improving people’s lives.
References:
1. Liu X, Deliu N, and Chakraborty B (2023). Micro-randomized trials: developing just-in-time adaptive interventions for better public health. American Journal of Public Health, 113(1): 60 – 69.
2. Aguilera A, Figueroa CA, Hernandez-Ramos R, Sarkar U, Cemballi AG, Gomez-Pathak L, Miramontes J, Avila-Garcia P, Tov EY, Chakraborty B, Yan X, Xu J, Modiri A, Aggarwal J, Williams JJ, and Lyles CR (2020). mHealth app using machine learning to increase physical activity in diabetes and depression: clinical trial protocol for the DIAMANTE study. BMJ Open, 10: e034723.
3. Figueroa C, Aguilera A, Chakraborty B, Modiri A, Aggarwal J, Deliu N, Sarkar U, Williams JJ, and Lyles CR (2021). Adaptive learning algorithms to optimize mobile applications for behavioral health: guidelines for design decisions. Journal of the American Medical Informatics Association, 28(6): 1225-1234.
4. Avalos MRA, Xu J, Figueroa CA, Haro-Ramos A, Chakraborty B, and Aguilera A (2024). The effect of cognitive behavioral therapy text messages on mood: A micro-randomized trial. PLOS Digital Health, DOI: 10.1371/journal.pdig.0000449.