Augmented outcome-weighted learning for estimating optimal dynamic treatment regimens.

TitleAugmented outcome-weighted learning for estimating optimal dynamic treatment regimens.
Publication TypeJournal Article
Year of Publication2018
AuthorsLiu, Ying, Yuanjia Wang, Michael R. Kosorok, Yingqi Zhao, and Donglin Zeng
JournalStat Med
Volume37
Issue26
Pagination3776-3788
Date Published2018 Nov 20
ISSN1097-0258
KeywordsAlgorithms, Clinical Protocols, Depressive Disorder, Major, Humans, Machine Learning, Models, Statistical, Outcome Assessment, Health Care, Precision Medicine
Abstract

Dynamic treatment regimens (DTRs) are sequential treatment decisions tailored by patient's evolving features and intermediate outcomes at each treatment stage. Patient heterogeneity and the complexity and chronicity of many diseases call for learning optimal DTRs that can best tailor treatment according to each individual's time-varying characteristics (eg, intermediate response over time). In this paper, we propose a robust and efficient approach referred to as Augmented Outcome-weighted Learning (AOL) to identify optimal DTRs from sequential multiple assignment randomized trials. We improve previously proposed outcome-weighted learning to allow for negative weights. Furthermore, to reduce the variability of weights for numeric stability and improve estimation accuracy, in AOL, we propose a robust augmentation to the weights by making use of predicted pseudooutcomes from regression models for Q-functions. We show that AOL still yields Fisher-consistent DTRs even if the regression models are misspecified and that an appropriate choice of the augmentation guarantees smaller stochastic errors in value function estimation for AOL than the previous outcome-weighted learning. Finally, we establish the convergence rates for AOL. The comparative advantage of AOL over existing methods is demonstrated through extensive simulation studies and an application to a sequential multiple assignment randomized trial for major depressive disorder.

DOI10.1002/sim.7844
Alternate JournalStat Med
Original PublicationAugmented outcome-weighted learning for estimating optimal dynamic treatment regimens.
PubMed ID29873099
PubMed Central IDPMC6191367
Grant ListR01 DK108073 / DK / NIDDK NIH HHS / United States
R01 NS073671 / NS / NINDS NIH HHS / United States
R01 GM124104 / GM / NIGMS NIH HHS / United States
R21 MH117458 / MH / NIMH NIH HHS / United States
P01 CA142538 / CA / NCI NIH HHS / United States
U01 NS082062 / NS / NINDS NIH HHS / United States
Project: