On the decision rules of cost-effective treatment for patients with diabetic foot syndrome

Abstract

Objective:

To assess the cost-effectiveness of two treatments (medical treatment and amputation) in patients with diabetic foot syndrome, one of the most disabling diabetic complications. Diabetes mellitus is a massive health care problem worldwide with a current prevalence of 150 millions diabetic cases, estimated to increase to 300 million cases in 2025.

Methods:

Integrating medical knowledge and advances into the clinical setting is often difficult due to the complexity of the algorithms and protocols. Clinical decision support systems assist the clinician in applying new information to patient care through the analysis of patient-specific clinical variables. We require strategic decision support to analyze the cost-effectiveness of these programs compared to the status quo. We provide a simple partially observable Markov model to investigate that issue, and we propose an heuristic algorithm to find the best policy of intervention.

Results:

This study assesses the potential cost-effectiveness of two alternative treatment interventions in patients with diabetic foot syndrome. The implementation of the heuristic algorithm solution will assist doctors in clinical decision making, and health care organizations in evaluating medication choices for effective treatment. Finally, our study reveals that treatment programs are highly cost-effective for patients at high risk of diabetic foot ulcers and lower extremity amputations.

Keywords:

• partially observable Markov decision model
• diabetic foot syndrome
• cost-effectiveness method

Introduction

An estimated 24 million Americans suffer from diabetes.¹ One of the major risk factors for lower extremity amputation is an antecedent foot ulcer. The symptoms of diabetic foot syndrome comprise a loss of protective sensation and, as a consequence, a lack of awareness, innervation of small muscles, and diminished fine control of pedal circulation. Polyneuropathy peripheral arterial occlusive disease, and infections, lead to diabetic foot syndrome. In most cases, the precipitant of ulceration is an accidental trauma which induces breaks of the skin and further complications such as ulcers, infections, or gangrene. Small bones may break due to loss of structural bone integrity.² About 15% of all diabetics require foot amputations, and 20%–25% of those even require re-amputations. The amputation aims at leaving a wound that will heal.³ Prevention programs and therapy at an early stage could diminish the number of amputations by as much as 80%–90%.^4,5

Researchers have applied several models to investigate the complications of the diabetic foot syndrome. Management of diabetic foot infections involves evaluating and determining the severity of infection, as the basis for selecting the appropriate approach to treatment.^6,7 Unfortunately, the lack of consensus on wound definitions and infection classification systems hampers the comparison of published studies. Consensus is developing that the key issues in classifying a diabetic foot wound are its depth (in particular, which tissues are involved) and whether the wound is complicated by either ischemia or infection.^8,9

Some researchers have analyzed the cost-effectiveness of treating foot infections in a diabetic cohort^10,11 by constructing a Markov model with Monte Carlo techniques. These studies combined multiple diabetic complications and analyzed prevention strategies for diabetes. The model predicted the rates of microvascular complications and mortality,¹² describing seven threats to validity for economic evaluations that are based on randomized controlled trials. Habacher et al¹³ provide a strong overview of the problems that may be encountered with modeling in economic evaluation.

In this paper, we construct and propose a partially observable Markov decision model and assess its implementation in the evaluation of diabetes mellitus prevention strategies for diabetic foot complications in Greece. Our paper is organized in the following sections:

First we present the proposed partially observable Markov decision process (POMDP) model, and we describe and discuss its underlying data. In the following section, we construct a suitable heuristic solution algorithm. The simplicity of the proposed algorithm gives the advantage of a supporting tool required for the prevention of diabetic foot syndrome (alongside clinical examination). In the subsequent section, we present the results of the model’s practical implementation in a case study. Finally, we discuss our findings and we make some suggestions for the practical utilization of the proposed model.

The POMDP model for the treatment of diabetic foot prevention

In this section we formulate a POMDP model¹⁴ in order to assess a cost-effectiveness analysis of prevention strategies for diabetic foot syndrome. Generally, POMDP models have been successfully reviewed for theory and for state-of-the-art solution algorithms.^15–18 To construct a Markov model for a particular application, one must first specify the Markov states. These health states should not only reflect all the relevant states of health associated with diabetes and treatment over time, but should also include and take into consideration all relevant clinical history. According to the international literature on the course of the disease (etiopathology), patients with diabetic foot syndrome pass through eight different stages: (S_i)^19,20

• no foot ulcer (S₁)

• uncomplicated ulcer (S₂)

• deep foot infection (S₃)

• foot ulcer and critical ischemia (S₄)

• primary healing (S₅)

• minor amputation (S₆)

• major amputation (S₇)

The second step in building a Markov model is to choose a cycle length, which must be a constant increment of time. So, we used the cycle length of half a year, because studies reported wound healing times of about six months for diabetic foot ulcers and amputations.^2,3,5 A POMDP model is typically defined as a six-parameter tuple. The six parameters together capture all the required aspects of medical decision making. A partially observable decision process is: $\mathit{POMDP}\hspace{0.17em}=\hspace{0.17em}<S,\hspace{0.17em}A,\hspace{0.17em}P,\hspace{0.17em}R,\hspace{0.17em}\mathrm{\Theta },\hspace{0.17em}q,\hspace{0.17em}\beta >$

• S is the set of physical states of a patient with diabetes mellitus. As mentioned above, we have seven states.

• A(t) ∈ A, decision taken at time t, t = 0, 1, 2, …

In our paper, we suggest that optimal prevention for high-risk patients is cost-saving, under the assumption of a 25% lower incidence of foot ulcers and extremity amputation compared to baseline prevention scenarios. (Also, we do not implement age- and risk group-specific transitions in our POMDP model). For simplicity, we consider two alternative treatments: medical treatments, and amputation, which are coded as 1 and 2 respectively; A = {1, 2}.

• Given transition probability matrices: $p_{\mathit{ij}}^k\hspace{0.17em}=\hspace{0.17em}\text{Pr}\hspace{0.17em}\left\{X(t)\hspace{0.17em}=\hspace{0.17em}j|X\hspace{0.17em}(t-1)\hspace{0.17em}=\hspace{0.17em}i,\hspace{0.17em}D(t-1)\hspace{0.17em}=\hspace{0.17em}k\right\}$(2.1)

In preparation for building the POMDP model, we are going to specify the transition probabilities with cohort studies in a central hospital in Athens.

• An observation on the system Y (t), and invariant conditional probabilities: $r_{i\theta }^k\hspace{0.17em}=\hspace{0.17em}\text{Pr}\hspace{0.17em}\left\{Y(t)\hspace{0.17em}=\hspace{0.17em}\theta |x(t)\hspace{0.17em}=\hspace{0.17em}j,\hspace{0.17em}d(t-1)\hspace{0.17em}=\hspace{0.17em}d\right\}$(2.2) Θ/j may also be assumed to be a continuous random variable, with a known conditional density f (θ/j). In this paper, we supposed that in a given time interval (check-up period), the physician counts the number of failed tests Y (t) = θ.

$r_{i\theta }\hspace{0.17em}=\hspace{0.17em}\frac{{\lambda }_i^{\theta }\cdot \hspace{0.17em}{e}^{-{\lambda }_i}}{\theta !},\hspace{0.17em}\theta \hspace{0.17em}=\hspace{0.17em}0,1,2,\dots$(2.3) where λ_i is the rate of failures tests in state i multiplied by the check-up time.¹⁷

Family physicians infrequently perform foot examinations in diabetic patients during routine office visits. Careful inspection of the diabetic foot on a regular basis is one of the easiest, least expensive, and most effective measures for preventing foot complications.⁷ Appropriate care of the diabetic foot requires recognition of the most common risk factors for limb loss.²¹ Many of these risk factors can be identified based on specific aspects of the clinical history, and a brief and systematic examination of the foot.

Common risk factors for amputation of the diabetic foot include peripheral neuropathy, structural foot deformity, ulceration, infection, and peripheral vascular disease. It is important to recognize that foot ulcers can have a multi-factorial etiology.²²

• Peripheral arterial occlusive disease typically involves the tibial and peroneal arteries, but spares the dorsalis pedis artery.²³

• The presence of lower extremity ischemia is suggested by a combination of clinical signs and symptoms, plus abnormal results on noninvasive vascular tests. Signs and symptoms may include claudication, and pain occurring in the arch or forefoot at rest or during the night.²¹

• Noninvasive vascular tests include transcutaneous oxygen measurement, absolute toe systolic pressure, and the ankle-brachial index (ABI). The ABI is a noninvasive test that can be performed easily in the office using a handheld Doppler device. The sensitivity and specificity of noninvasive vascular tests are a matter of some controversy.²⁴

• Digital symmetric polyneuropathy is perhaps the most common complication affecting the lower extremities of patients with diabetes mellitus.⁴

• The nylon monofilament test is a simply-performed office test to diagnose patients at risk for ulcer formation due to peripheral sensory neuropathy. The test is abnormal if the patient cannot sense the touch of the monofilament when it is pressed against the foot with just enough pressure to bend the filament.

• Foot deformities, which are common in diabetic patients, lead to focal areas of high pressure. When an abnormal focus of pressure is coupled with lack of sensation, a foot ulcer can develop.²³

• Also, a diabetic patient with a history of previous ulceration is at increased risk for further ulceration, infection, and subsequent amputation.

The probabilistic relation between observation process Y(t) and core process X(t) is given by the 7 × 7 time-invariant observation matrix R. We apply the expectation-maximization (EM) algorithm¹⁸ to estimate the above observation probabilities. Alternatively, we can apply cohort studies.²²

• Given real values q(i, k), which signify the expected incurred immediate costs/rewards in taking action k, while being in patient state i.

• We have a discount factor β, 0 < β < 1.

In a POMDP model, the state of the patient is not known with certainty. The vector of state probabilities π(t) = (π₁(t), (π₂(t), …, (π₇(t)) is called the information vector: $\begin{array}{l}\begin{array}{lll}{\pi }_i(t)\hfill & =\hfill & \text{Pr}\hspace{0.17em}\{X(t)\hspace{0.17em}=\hspace{0.17em}i\hspace{0.17em}|Y(t),\hspace{0.17em}A(t-1),\hspace{0.17em}Y(t-1),\hspace{0.17em}\dots ,\hfill \\ \hfill & \hfill & A(1),\hspace{0.17em}Y(1),\hspace{0.17em}A(0)\}\hfill \end{array}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\end{array}$(2.4)

It is well known that π(t) is a sufficient statistic.²⁵ More precisely, π(t) summarizes all of the necessary information on the history of the process for choosing an action at time t. The probabilities π_i(t) in the type (2.4) are updated and reassigned to the system states by the decision maker at every step t, given the history of the process. However, the update of π_i(t) is Bayesian, such that the detail of the history is not used directly. Specifically, ${\pi }_i\hspace{0.17em}(t)\hspace{0.17em}=\hspace{0.17em}\frac{r_{i\theta }^{\alpha }\cdot \hspace{0.17em}\pi (t-1)\cdot P_i^{A(t-1)}}{\sum _{l=1}^7{r}_{l\theta }^{\alpha }\hspace{0.17em}\cdot \hspace{0.17em}\pi (t-1)\hspace{0.17em}\cdot \hspace{0.17em}{P}_l^{A(t-1)}}$(2.5a) ${\pi }_i\hspace{0.17em}(t)\hspace{0.17em}=\hspace{0.17em}\frac{f\hspace{0.17em}(y|i)\hspace{0.17em}\cdot \hspace{0.17em}\pi (t-1)\hspace{0.17em}\cdot \hspace{0.17em}{P}_i^{A(t-1)}}{\sum _{l=1}^{7}f\hspace{0.17em}(y|i)\hspace{0.17em}\cdot \hspace{0.17em}\pi (t-1)\cdot \hspace{0.17em}{P}_l^{A(t-1)}},$(2.5b) where, P_l is the first column of P, and matrix multiplication is implied where appropriate. To clarify, the order of things in the decision process is the following:

At time t, the state of a patient with diabetes mellitus is an unknown state X(t) = i. Based on π(t – 1), the matrices P, and the vectors q(·, A(t – 1)) a decision is taken: A(t) = k, k = 1, 2. Instantaneously, the patient switches to unknown state j. According to $p_{ij}^k$, an immediate average reward or cost q(i, k) is gained, and the observation process provides a signal Y(t) = θ. The decision maker then updates the state probabilities by (2.5a) or (2.5b). The process repeats itself indefinitely. Focusing on the total discounted reward criterion, we wish to optimize the decisions so as to maximize: $E\left[\sum _{t=0}^{\infty }{\beta }^t\hspace{0.17em}\cdot \hspace{0.17em}q\left(X\hspace{0.17em}\left(t\right),\hspace{0.17em}A\left(t\right)\right)\right]$(2.6) for all initial probability vectors π (0). It is assumed that the process is initiated with a known probability distribution over the state of the patient. The dependence on π(0) has been dropped from the notation in (2.6), and 0 < β < 1 is a discount factor.

A solution method

The generality of the standard POMDP model¹⁴ limits practical application of the framework, due to the computational complexity of associated solution methods. However, for many specialized problems, the full-blown generality of the POMDP approach and its associated solution methods is superfluous. In this section, we provide a heuristic rule that it is stationary and extremely simple. It runs as follows:

Step 1

We solve the fully observable problem by Howard’s policy iteration,²⁵ and then we value the determination routine. We find d^*(i) for all 1 ≤ i ≤ 7, where d^*(i) stands for the Howard’s optimal decision, assuming state i is fully observed, d(i) ∈ D. As an output of the algorithm, we get the V_i values; the expected total discounted reward starting in state i.

Notably: $V_i\hspace{0.17em}=\hspace{0.17em}\underset{\alpha }{\text{max}}\left[q\left(i,\hspace{0.17em}\alpha \right)\hspace{0.17em}+\hspace{0.17em}\beta \sum _{j=1}^n{p}_{ij}^k\cdot V_j\right]$(3.1)

Step 2

For the current state vector π(t), we calculate an approximate reward functional Ṽ(π(t)) $\tilde{V}\hspace{0.17em}\left(\pi \left(t\right)\right)\hspace{0.17em}=\hspace{0.17em}\text{max}\left\{\sum _{i=1}^7{\pi }_i\left(t\right)\hspace{0.17em}\cdot \hspace{0.17em}\left(q\left(i,\hspace{0.17em}\alpha \right)\hspace{0.17em}+\hspace{0.17em}\beta \sum _{j=1}^n{p}_{ij}^{\alpha }\cdot V_j\right)|\alpha =1,\hspace{0.17em}2\right\}$(3.2) and set A(t) = α̃ = arg max Ṽ(π(t)). The advantage of the above algorithm is the fact that it requires only a single run of Howard’s routine.

Implementation of the proposed model – a case study

For comprehension of the above algorithm, we present the implementation results from the treatment of patients with diabetic foot syndrome in a middle-sized public hospital in Greece, during the year 2008. We have used seven health states S = {1, 2, 3, …, 7}, and two treatments A = {1, 2}. $P^1\hspace{0.17em}=\hspace{0.17em}\left(\begin{array}{lllllll}0.15\hfill & 0.10\hfill & 0.15\hfill & 0.15\hfill & 0.25\hfill & 0.10\hfill & 0.10\hfill \\ 0.20\hfill & 0.10\hfill & 0.10\hfill & 0.10\hfill & 0.20\hfill & 0.10\hfill & 0.20\hfill \\ 0.00\hfill & 0.15\hfill & 0.20\hfill & 0.15\hfill & 0.30\hfill & 0.10\hfill & 0.10\hfill \\ 0.00\hfill & 0.00\hfill & 0.10\hfill & 0.30\hfill & 0.10\hfill & 0.20\hfill & 0.10\hfill \\ 0.00\hfill & 0.00\hfill & 0.10\hfill & 0.20\hfill & 0.40\hfill & 0.10\hfill & 0.20\hfill \\ 0.00\hfill & 0.00\hfill & 0.00\hfill & 0.50\hfill & 0.20\hfill & 0.10\hfill & 0.20\hfill \\ 0.00\hfill & 0.00\hfill & 0.00\hfill & 0.40\hfill & 0.10\hfill & 0.20\hfill & 0.30\hfill \end{array}\right)$ $P^2\hspace{0.17em}=\hspace{0.17em}\left(\begin{array}{lllllll}0.30\hfill & 0.10\hfill & 0.20\hfill & 0.20\hfill & 0.10\hfill & 0.10\hfill & 0.00\hfill \\ 0.10\hfill & 0.20\hfill & 0.25\hfill & 0.25\hfill & 0.15\hfill & 0.05\hfill & 0.00\hfill \\ 0.15\hfill & 0.30\hfill & 0.20\hfill & 0.15\hfill & 0.10\hfill & 0.10\hfill & 0.00\hfill \\ 0.30\hfill & 0.30\hfill & 0.20\hfill & 0.10\hfill & 0.05\hfill & 0.00\hfill & 0.05\hfill \\ 0.30\hfill & 0.30\hfill & 0.10\hfill & 0.10\hfill & 0.05\hfill & 0.10\hfill & 0.05\hfill \\ 0.20\hfill & 0.10\hfill & 0.30\hfill & 0.10\hfill & 0.10\hfill & 0.10\hfill & 0.10\hfill \\ 0.20\hfill & 0.20\hfill & 0.10\hfill & 0.20\hfill & 0.10\hfill & 0.10\hfill & 0.10\hfill \end{array}\right)$

The rewards/costs are derived from the National Statistical Service of Greece. The direct costs include drug costs, hospitalization costs, monitoring costs, and physician visit costs. $\begin{array}{ll}q(1,\hspace{0.17em}1)\hspace{0.17em}=\hspace{0.17em}96€\hfill & q(1,\hspace{0.17em}2)\hspace{0.17em}=\hspace{0.17em}1330€\hfill \\ q(2,\hspace{0.17em}1)\hspace{0.17em}=\hspace{0.17em}358€\hfill & q(2,\hspace{0.17em}2)\hspace{0.17em}=\hspace{0.17em}1500€\hfill \\ q(3,\hspace{0.17em}1)\hspace{0.17em}=\hspace{0.17em}370€\hfill & q(3,\hspace{0.17em}2)\hspace{0.17em}=\hspace{0.17em}2000€\hfill \\ q(4,\hspace{0.17em}1)\hspace{0.17em}=\hspace{0.17em}450€\hfill & q(4,\hspace{0.17em}2)\hspace{0.17em}=\hspace{0.17em}3000€\hfill \\ q(5,\hspace{0.17em}1)\hspace{0.17em}=\hspace{0.17em}500€\hfill & q(5,\hspace{0.17em}2)\hspace{0.17em}=\hspace{0.17em}3300€\hfill \\ q(6,\hspace{0.17em}1)\hspace{0.17em}=\hspace{0.17em}600€\hfill & q(6,\hspace{0.17em}2)\hspace{0.17em}=\hspace{0.17em}8000€\hfill \\ q(7,\hspace{0.17em}1)\hspace{0.17em}=\hspace{0.17em}772€\hfill & q(7,\hspace{0.17em}2)\hspace{0.17em}=\hspace{0.17em}11000€\hfill \end{array}$

The costs for the diabetic foot syndrome, and the model’s effectiveness parameters were taken from various studies.^20,26–28 We considered a discount factor β=0.90. When estimating transition probabilities, it is important to be mindful of the cohorts or target populations that are to be analyzed, as one can have a number of different transition probability matrices that are specific for different cohorts, defined by demographic or clinical variables. In order to find the final information vector, we can use regression models.

For example, if the initial information vector for a patient with diabetic foot syndrome is π = (0.10, 0.20, 0.10, 0.10, 0.30, 0.20, 0.00), using Howard’s algorithm, we have: $\begin{array}{l}{V}_1\hspace{0.17em}=\hspace{0.17em}296\hfill \\ V_2\hspace{0.17em}=\hspace{0.17em}454\hfill \\ V_3\hspace{0.17em}=\hspace{0.17em}1457\hfill \\ V_4\hspace{0.17em}=\hspace{0.17em}3252\hfill \\ V_5\hspace{0.17em}=\hspace{0.17em}3496\hfill \\ V_6\hspace{0.17em}=\hspace{0.17em}5722\hfill \\ V_7\hspace{0.17em}=\hspace{0.17em}8976\hfill \end{array}$ Ṽ (π) = max {3723, 3970} = 3970, and the optimal decision is amputation: Ṽ^* (5) = 2. To clarify, the order of things in the decision process according above POMDP model is the following. At time t = 0, a patient with diabetic foot syndrome has an information vector π(0) = (0.10, 0.20, 0.20, 0.20, 0.30, 0.00, 0.00). This information vector is a sufficient statistic for taking an action.²⁵ In terms of the above model, we take amputation after the state “primary healing”. So, the doctor has a supporting tool in their decision making.

Discussion

The risk of injury to the diabetic foot is high, and the consequences of injury can be severe. Therefore, prevention based upon clinical trials and doctors is undoubtedly the best strategy. Decision analytic modeling, undertaken for the purpose of economic evaluation of health technologies, involves the application of mathematical techniques to synthesize available information about health care processes and their implications.^10,15,20

In our paper, we provide a simple POMDP model for taking a cost-effective decision for the treatment of foot ulcer syndrome. Many researchers have proposed similar models for the management of diabetic foot ulcer.^13,19,29 However, in these models, the integration of medical knowledge and advances into the clinical setting is often difficult, due to the complexity of the algorithms and protocols. So, in our model, we propose the incorporation of the required information into a Clinical Decision Support System (CDSS), in order to analyze the patient-specific clinical variables. In this way, we can essentially assist the clinician toward their best medical decision making. In our proposed model, we also used cost-related information, incorporating utility functions, to support the cost-effectiveness evaluation of these programs, and the strategic management of health care organizations.

So, the main advantage of the proposed model lies in the clinician’s capability for full adaptation control, meaning ease of modification of any of the model’s parameters incorporated into the CDSS.

Conclusion

This model is significant because it provides an explicit two-way bridge between primary clinical data and early and efficient medical decision making. Typically, the objective of this decision model is to obtain a clearer understanding of the relationship between incremental costs and their consequences.

The proposed simple model (incorporated into a CDSS) results in health care quality improvement, cost-effective clinical decision making, and adaptability and transferability through different health care environments (either organizations or countries). So, the exchange of reliable and timely information through various levels of the health care system in Greece will facilitate the efficiency and effectiveness of the proposed model.

Acknowledgements

The authors gratefully acknowledge the reviewers of this manuscript for their invaluable comments and suggestions.

Disclosure

The authors report no conflict of interest in this work.