Thompson sampling for count data in mobile health trials (Invited Talk @ University of Cambridge)

Abstract

Mobile health (mHealth) technologies aim to improve distal outcomes, such as clinical conditions, by optimizing proximal outcomes through just-in-time adaptive interventions. Contextual bandits provide a suitable framework for customizing such interventions, however, unique challenges such as modeling count outcomes persist within existing bandit frameworks. We address this challenge by leveraging count data models into online bandit approaches for micro-randomized trials. Specifically, we focus on a popular contextual bandit algorithm called Thompson sampling. We present theoretical results on our proposed strategies, as well as empirical results both in simulations and on a real dataset from the Drink Less trial. While the focus of this work is on design aspects, we conclude by discussing the inference problem in adaptively-collected data through a novel statistical test based on the allocation probabilities.