Current recommendations on the estimation of transition probabilities in Markov cohort models for use in health care decision-making: a targeted literature review

Figures & data

Figure 1 Illustrative example of a three-state Markov model.

Table 1 HTA agency websites considered

Country	HTA agency
Australia	Pharmaceutical Benefits Advisory Committee (PBAC)
Belgium	Belgian Health Care Knowledge Centre (KCE)
Canada	Canadian Agency for Drugs and Technologies in Health (CADTH)
France	The French National Authority for Health (HAS)
Germany	Institute for Quality and Efficiency in Health Care (IQWiG)
Ireland	National Centre for Pharmacoeconomics (NCPE)
Norway	The Norwegian Medicines Agency (NoMA)
Portugal	National Authority of Medicines and Health Products (INFARMED)
Sweden	Dental and Pharmaceutical Benefits Agency (TLV)
UK*	National Institute for Health and Care Excellence (NICE), Scottish Medicines Consortium (SMC), All Wales Medicines Strategy Group (AWMSG)

Note:

* Including NICE Decision Support Unit reports.

Abbreviation: HTA, health technology assessment.

Table 2 Results of the electronic database search

Electronic database	Records identified	After deduplication
Medline	1,057	1,043
Embase	1,422	775
PubMed	1,098	111
The Cochrane Library	59	2
Total	3,636	1,931

Figure 2 Flow diagram of the screening process for the electronic database search.

Figure S1 Search strategy.

Table S1 Potentially relevant publications identified via the electronic searches

Reference	Title
Aalen et al^Citation1	Covariate adjustment of event histories estimated from Markov chains: the additive approach
Alam et al^Citation2	Investigating the impact of structural changes in a nice single technology appraisal cost-effectiveness model
Alarid-Escudero et al^Citation3	Calibration of piecewise Markov models using a change-point analysis through an iterative convex optimization algorithm
Albert and Waclawiw^Citation4	A two-state Markov chain for heterogeneous transitional data: a quasi-likelihood approach
Allignol et al^Citation5	A competing risks approach for nonparametric estimation of transition probabilities in a non-Markov illness-death model
Andersen and Pohar Perme^Citation6	Inference for outcome probabilities in multi-state models
Black et al^Citation7	Determining transition probabilities from mortality rates and autopsy findings
Borgan^Citation8	Estimation of covariate-dependent Markov transition probabilities from nested case-control data. (Erratum is reference 9)
Borgan^Citation9	Erratum: Estimation of covariate-dependent Markov transition probabilities from nested case-control data (Original paper is reference 8)
Boruvka and Cook^Citation10	Sieve estimation in a Markov illness-death process under dual censoring
Cohen^Citation11	On estimating the equilibrium and transition probabilities of a finite-state Markov chain from the same data
Cooper et al^Citation12	Comprehensive decision analytical modelling in economic evaluation: a Bayesian approach
Craig and Sendi^Citation13	Estimation of the transition matrix of a discrete-time Markov chain
Dabrowska and Ho^Citation14	Confidence bands for comparison of transition probabilities in a Markov chain model
Debosscher^Citation15	Unifying stochastic Markov process and its transition probability density function
Denton and Spencer^Citation16	Modeling the age dynamics of chronic health conditions: life-table-consistent transition probabilities and their application
Diaby et al^Citation17	Survival modeling for the estimation of transition probabilities in model-based economic evaluations in the absence of individual patient data: a tutorial
Gaveau and Schulman^Citation18	Multiple phases in stochastic dynamics: geometry and probabilities
Gupta et al^Citation19	Transition probability estimation using repeated sampling from a fitted mixed model
Gupta et al^Citation20	Generalized implementation of Em algorithm for estimation of transition probability matrix
Hawkins and Han^Citation21	Estimating transition probabilities from aggregate samples plus partial transition data
Healy and Engler^Citation22	Modeling disease-state transition heterogeneity through Bayesian variable selection
Huang^Citation23	Integrated analysis of incidence, progression, regression and disappearance probabilities
Iacobelli and Carstensen^Citation24	Multiple time scales in multi-state models
Kassteele et al^Citation25	Estimating net transition probabilities from cross-sectional data with application to risk factors in chronic disease modeling
Kaushik et al^Citation26	A methodology to monitor the changing trends in health status of an elderly person by developing a Markov model
Li and Pack^Citation27	An application of Markov models in estimating transition probabilities for postmenopausal women with osteoporosis
Limwattananon and Limwattananon^Citation28	Constructing a state-transition model for an economic evaluation of cancer treatments
McCombs^Citation29	Pharmacoeconomics: what is it and where is it going?
Meidani and Ghanem^Citation30	Uncertainty quantification for Markov chain models
Meira-Machado et al^Citation31	Nonparametric estimation of transition probabilities in a non-Markov illness-death model
Meira-Machado et al^Citation32	Multi-state models for the analysis of time-to-event data
Miller and Homan^Citation33	Determining transition probabilities: confusion and suggestions
Milne^Citation34	Pharmacoeconomic models in disease management. A guide for the novice or the perplexed
Moussa et al^Citation35	Measuring the change in contingency tables using Markov models application to the effect of preceding conception on the next one
Muenz and Rubinstein^Citation36	Markov models for covariate dependence of binary sequences
Nagylaki^Citation37	The distribution of sojourn times in finite absorbing Markov chains
Neine et al^Citation38	Bayesian calibration method to estimate transition probabilities for a Markov model based on a continuous outcome measure: application in Parkinson’s disease
Ng and Cook^Citation39	Modeling two-state disease processes with random effects
O’Mahony et al^Citation40	Dealing with time in health economic evaluation: methodological issues and recommendations for practice
Oppe et al^Citation41	Comparing methods of data synthesis: re-estimating parameters of an existing probabilistic cost-effectiveness model
Paes and Lima^Citation42	A SAS macro for estimating transition probabilities in semiparametric models for recurrent events
Putter et al^Citation43	Tutorial in biostatistics: competing risk and multi-state models
Regnier and Shechter^Citation44	State-space size considerations for disease-progression models
Rodriguez-Girondo and Una-Alvarez^Citation45	A nonparametric test for Markovianity in the illness-death model
Rodriguez-Girondo and Una-Alvarez^Citation46	Testing Markovianity in the three-state progressive model via future-past association
Rodriguez-Girondo and Una-Alvarez^Citation47	Methods for testing the Markov condition in the illness-death model: a comparative study
Rosychuk et al^Citation48	Comparison of variance estimation approaches in a two-state Markov model for longitudinal data with misclassification
Rosychuk and Thompson^Citation49	Bias correction of two-state latent Markov process parameter estimates under misclassification
Saint-Pierre et al^Citation50	The analysis of asthma control under a Markov assumption with use of covariates
Spitoni et al^Citation51	Estimation and asymptotic theory for transition probabilities in Markov renewal multi-state models
Tattar and Vaman^Citation52	Testing transition probability matrix of a multi-state model with censored data
Tattar and Vaman^Citation53	The k-sample problem in a multi-state model and testing transition probability matrices
Titman^Citation54	Transition probability estimates for non-Markov multi-state models
Tsang et al^Citation55	Estimating Markov chain transition matrices in limited data samples: a Monte Carlo experiment
Welton and Ades^Citation56	Estimation of Markov chain transition probabilities and rates from fully and partially observed data: uncertainty propagation, evidence synthesis, and model calibration

AalenOOBorganOFekjaerHCovariate adjustment of event histories estimated from Markov chains: the additive approach.[Erratum appears in Biometrics. 2003 Jun;59(2):452–453]Biometrics2001574993100111764270

 PubMed Web of Science ®Google Scholar

AlamMFBartonPMonahanMInvestigating the impact of structural changes in a NICE single technology appraisal cost-effectiveness modelValue Health2015187A696

 Web of Science ®Google Scholar

Alarid-EscuderoFEnnsEPeralta-TorresYEMaclehoseRKuntzKMCalibration of piecewise Markov models using a change-point analysis through an iterative convex optimization algorithmValue Health2015187A814

 Web of Science ®Google Scholar

AlbertPSWaclawiwMAA two-state Markov chain for heterogeneous transitional data: a quasi-likelihood approachStat Med19981713148114939695193

 PubMed Web of Science ®Google Scholar

AllignolABeyersmannJGerdsTLatoucheAA competing risks approach for nonparametric estimation of transition probabilities in a non-Markov illness-death modelLifetime Data Anal201420449551323807694

 PubMed Web of Science ®Google Scholar

AndersenPKPohar PermeMInference for outcome probabilities in multi-state modelsLifetime Data Anal200814440543118791824

 PubMed Web of Science ®Google Scholar

BlackWCNeaseRFJrWelchHGDetermining transition probabilities from mortality rates and autopsy findingsMed Decis Making199717187938994155

 PubMed Web of Science ®Google Scholar

BorganOEstimation of covariate-dependent Markov transition probabilities from nested case-control dataStat Methods Med Res200211218320212040696

 PubMed Web of Science ®Google Scholar

BorganOErratum: Estimation of covariate-dependent Markov transition probabilities from nested case-control dataStat Methods Med Res2003122124

 Web of Science ®Google Scholar

BoruvkaACookRJSieve estimation in a Markov illness-death process under dual censoringBiostatistics201617235036326598559

 PubMed Web of Science ®Google Scholar

CohenJEOn estimating the equilibrium and transition probabilities of a finite-state Markov chain from the same dataBiometrics19682411851875642403

 PubMed Web of Science ®Google Scholar

CooperNJSuttonAJAbramsKRTurnerDWailooAComprehensive decision analytical modeling in economic evaluation: a Bayesian approachHealth Econ200413320322614981647

 PubMed Web of Science ®Google Scholar

CraigBASendiPPEstimation of the transition matrix of a discrete-time Markov chainHealth Econ2002111334211788980

 PubMed Web of Science ®Google Scholar

DabrowskaDMHoWTConfidence bands for comparison of transition probabilities in a Markov chain modelLifetime Data Anal20006152110763558

 PubMed Web of Science ®Google Scholar

DebosscherAUnifying stochastic Markov process and its transition probability density functionPhys Rev A19914412792979389905942

 PubMed Web of Science ®Google Scholar

DentonFTSpencerBGModeling the age dynamics of chronic health conditions: life-table-consistent transition probabilities and their applicationCan J Aging201534217619325917602

 PubMed Web of Science ®Google Scholar

DiabyVAdunlinGMonteroAJSurvival modeling for the estimation of transition probabilities in model-based economic evaluations in the absence of individual patient data: a tutorialPharmacoeconomics201432210110824338265

 PubMed Web of Science ®Google Scholar

GaveauBSchulmanLSMultiple phases in stochastic dynamics: geometry and probabilitiesPhys Rev E Stat Nonlin Soft Matter Phys2006733 Pt 203612416605615

 PubMed Web of Science ®Google Scholar

GuptaSBhattacharyyaSSonathiVBakuliAMathurAKLeteneuxCTransition probability estimation using repeated sampling from a fitted mixed modelValue Health2014177A326

 Web of Science ®Google Scholar

GuptaSChattopadhyaySGundaPGeneralized implementation of Em algorithm for estimation of transition probability matrixValue Health2015187A694

 Web of Science ®Google Scholar

HawkinsDLHanCPEstimating transition probabilities from aggregate samples plus partial transition dataBiometrics200056384885410985225

 PubMed Web of Science ®Google Scholar

HealyBCEnglerDModeling disease-state transition heterogeneity through Bayesian variable selectionStat Med20092891353136819206077

 PubMed Web of Science ®Google Scholar

HuangGHIntegrated analysis of incidence, progression, regression and disappearance probabilitiesBMC Med Res Methodol200884018577235

 PubMed Web of Science ®Google Scholar

IacobelliSCarstensenBMultiple time scales in multi-state modelsStat Med201332305315532724027131

 PubMed Web of Science ®Google Scholar

KassteeleJvHoogenveenRTEngelfrietPMBaalPHBoshuizenHCEstimating net transition probabilities from cross-sectional data with application to risk factors in chronic disease modelingStat Med201231653354322139860

 PubMed Web of Science ®Google Scholar

KaushikACellerBGAmbikairajahEA methodology to monitor the changing trends in health status of an elderly person by developing a Markov modelConf Proc IEEE Eng Med Biol Soc200522171217417282661

 PubMedGoogle Scholar

LiZPackSAn application of Markov models in estimating transition probabilities for postmenopausal women with osteoporosisDrug Info J20043814146

 Google Scholar

LimwattananonCLimwattananonSConstructing a state-transition model for an economic evaluation of cancer treatmentsJ Med Assoc Thai201497Suppl 5S108S112

 Google Scholar

McCombsJSPharmacoeconomics: what is it and where is it going?Am J Hypertens1998118 Pt 2112S119S discussion 135S–137S9717852

 PubMed Web of Science ®Google Scholar

MeidaniHGhanemRUncertainty quantification for Markov chain modelsChaos201222404310223278037

 PubMed Web of Science ®Google Scholar

Meira-MachadoLde Una-AlvarezJCadarso-SuarezCNonparametric estimation of transition probabilities in a non-Markov illness-death modelLifetime Data Anal200612332534416917736

 PubMed Web of Science ®Google Scholar

Meira-MachadoLde Una-AlvarezJCadarso-SuarezCAndersenPKMulti-state models for the analysis of time-to-event dataStat Methods Med Res200918219522218562394

 PubMed Web of Science ®Google Scholar

MillerDKHomanSMDetermining transition probabilities: confusion and suggestionsMed Decis Making199414152588152357

 PubMed Web of Science ®Google Scholar

MilneRJPharmacoeconomic models in disease management. A guide for the novice or the perplexedDis Manag Health Outcomes199843119134

 Google Scholar

MoussaMAHathoutHKasraouiRSalehAKMeasuring the change in contingency tables using Markov models application to the effect of preceding conception on the next oneMethods Inf Med198221131337057716

 PubMed Web of Science ®Google Scholar

MuenzLRRubinsteinLVMarkov models for covariate dependence of binary sequencesBiometrics1985411911014005390

 PubMed Web of Science ®Google Scholar

NagylakiTThe distribution of sojourn times in finite absorbing Markov chainsMath Biosci1976281–26972

 Google Scholar

NeineMBriquetBMokdadCEVataireALAballeaSBayesian calibration method to estimate transition probabilities for a Markov model based on a continuous outcome measure: application in Parkinson’s diseaseValue Health2013167A325A326

 Web of Science ®Google Scholar

NgETCookRJModeling two-state disease processes with random effectsLifetime Data Anal1997343153359503879

 PubMedGoogle Scholar

O’MahonyJFNewallATvan RosmalenJDealing with time in health economic evaluation: methodological issues and recommendations for practicePharmacoeconomics201533121255126826105525

 PubMed Web of Science ®Google Scholar

OppeMAlMRutten-van MolkenMComparing methods of data synthesis: re-estimating parameters of an existing probabilistic cost-effectiveness modelPharmacoeconomics201129323925021142288

 PubMed Web of Science ®Google Scholar

PaesATde LimaACA SAS macro for estimating transition probabilities in semiparametric models for recurrent eventsComput Methods Programs Biomed2004751596515158048

 PubMed Web of Science ®Google Scholar

PutterHFioccoMGekusRBTutorial in biostatistics: competing risk and multi-state modelsStat Med200726112389243017031868

 PubMed Web of Science ®Google Scholar

RegnierEDShechterSMState-space size considerations for disease-progression modelsStat Med201332223862388023609629

 PubMed Web of Science ®Google Scholar

Rodriguez-GirondoMde Una-AlvarezJA nonparametric test for Markovianity in the illness-death modelStat Med201231304416442722975898

 PubMed Web of Science ®Google Scholar

Rodriguez-GirondoMde Una-AlvarezJTesting Markovianity in the three-state progressive model via future-past associationBiom J201254216318022522376

 PubMed Web of Science ®Google Scholar

Rodriguez-GirondoMUna-AlvarezJdMethods for testing the Markov condition in the illness-death model: a comparative studyStat Med201635203549356226990971

 PubMed Web of Science ®Google Scholar

RosychukRJShengXStuberJLComparison of variance estimation approaches in a two-state Markov model for longitudinal data with misclassificationStat Med200625111906192116220512

 PubMed Web of Science ®Google Scholar

RosychukRJThompsonMEBias correction of two-state latent Markov process parameter estimates under misclassificationStat Med200322122035205512802821

 PubMed Web of Science ®Google Scholar

Saint-PierrePCombescureCDauresJPGodardPThe analysis of asthma control under a Markov assumption with use of covariatesStat Med200322243755377014673936

 PubMed Web of Science ®Google Scholar

SpitoniCVerduijnMPutterHEstimation and asymptotic theory for transition probabilities in Markov renewal multi-state modelsInt J Biostat2012812322944722

 PubMedGoogle Scholar

TattarPNVamanHJTesting transition probability matrix of a multi-state model with censored dataLifetime Data Anal200814221623017874296

 PubMed Web of Science ®Google Scholar

TattarPNVamanHJThe k-sample problem in a multi-state model and testing transition probability matricesLifetime Data Anal201420338740323722306

 PubMed Web of Science ®Google Scholar

TitmanACTransition probability estimates for non-Markov multi-state modelsBiometrics20157141034104126148652

 PubMed Web of Science ®Google Scholar

TsangKPWangBCMGarrisonLEstimating Markov chain transition matrices in limited data samples: A Monte Carlo experimentValue Health2012154A164A165

 Web of Science ®Google Scholar

WeltonNJAdesAEEstimation of Markov chain transition probabilities and rates from fully and partially observed data: uncertainty propagation, evidence synthesis, and model calibrationMed Decis Making200525663364516282214

 PubMed Web of Science ®Google Scholar