π€ ML: Statistical reinforcement learning and multi-armed bandits
Nov 6, 2026
How do we learn, while people use an app, take a course or follow a treatment, which option works best for whom, and when? I develop reinforcement-learning and multi-armed bandit methods that are statistically considerate: they optimise outcomes while still allowing valid scientific conclusions, often within a Bayesian framework (e.g. Thompson sampling).
Applications:biostatistics (just-in-time adaptive interventions in mobile health, dynamic treatment regimes, SMARTs); education (adaptive experiments in online courses); finance (sequential decision-making under uncertainty).
Team:Bibhas Chakraborty (Duke-NUS, Singapore);
Sofia S. Villar (MRC Biostatistics Unit, University of Cambridge);
Joseph Jay Williams (University of Toronto); the Google team behind PEARL, including Amy Armento Lee (PhD student I co-supervise, University of Cambridge);
Caroline A. Figueroa (DIAMANTE study); my PhD student Simone Cuonzo (Sapienza).
Simone Cuonzo, Nina Deliu
(2026).
A Latent-State-Aware Framework for Non-Stationary UCB Bandits.
In Proceedings of the Statistics and Data Science Workshop 2026, Caserta (in press).
Ilya Musabirov, Mohi Reza, Steven Moore, Pan Chen, Harsh Kumar, Tong Li, Haochen Song, Jiakai Shi, Koby Choy, Thomas Price, John Stamper, Norman Bier, Nina Deliu, Sofia S Villar, Anna Rafferty, Audrey Durand, Joseph J Williams
(2024).
Platform-based Adaptive Experimental Research in Education: Lessons Learned from Digital Learning Challenge.
In Proceedings of the 14th International Conference on Learning Analytics & Knowledge (LAK24). ACM.
Nina Deliu
(2022).
Multinomial Thompson Sampling for adaptive experiments with rating scales.
In Book of Short Papers SIS 2022. Pearson, pp. 1065β1070.