Examining and interpreting responsiveness of the Diabetes Medication Satisfaction measure

Abstract

Objective: Treatment satisfaction (TS) is an important patient reported outcome (PRO) in diabetes as it is correlated with outcomes necessary for optimal treatment (e.g., compliance, self-management behaviour). The objective of this study was to examine the responsiveness of the DiabMedSat, a disease-specific PRO measure, assessing Overall, Burden, Efficacy and Symptom TS.

Methods: The DiabMedSat was included in an open label, observational study of the safety and efficacy of biphasic insulin aspart 30 (NovoMix 30) in routine practice with type 2 diabetes. Responsiveness analyses, examining both internal and external responsiveness, were conducted and minimally important differences (MID) assessed.

Results: In 18,817 patients, all TS scores significantly improved after 26 weeks of treatment (p<0.001). The effect sizes for these changes were above 0.5 indicating that the ability to detect change was moderate-to-large in size. Significant differences were found for all TS scores comparing patients who met their HbA_1c goal, who improved but did not meet goal and who did not improve (p<0.01), and for patients who experienced a minor hypoglycaemic event and those who did not (p<0.001). DiabMedSat scores were able to detect changes in patients’ own global rating of satisfaction (MID ranging from 5.3 to 11.7) and in physician-rated satisfaction with patients’ HbA_1c improvement (MID ranging from 5.3 to 10.2).

Conclusions: In the context of an observational study, the DiabMedSat has been shown to be highly responsive to change and can be considered as an acceptable PRO measure for TS in diabetes.

Key words::

• blood glucose/drug effects
• diabetes mellitus
• type 2, insulin
• Asp(B28)
• patient satisfaction/statistics
• psychometrics/instrumentation
• questionnaires
• treatment outcomes

Introduction

Diabetes is a debilitating, complex, common chronic illness requiring lifelong self-management. Disease management is a constant challenge to physicians and patients as poor adherence to diabetes treatment ultimately leads to negative consequences. Thus, treatment satisfaction (TS) is an important patient reported outcome (PRO) as it may be correlated with impaired self-management¹, patient compliance² and improvements in glycaemic control³. Treatment satisfaction may account for many of the preferences patients have for a particular insulin delivery system⁴. Assessing TS can help the physician distinguish among treatment regimes with equal efficacy⁵, as well as identify treatments which have greater effectiveness¹.

Responsiveness is the extent to which a health status measure accurately reflects change in a patient's condition over time^{6, 7} and requires longitudinal data to examine this change as a result of treatment. Responsiveness is a key psychometric property as a PRO measure may fail to detect change even when true change has occurred if the responsiveness of the measure is not adequate⁸. Further, positive responsiveness findings help confirm that the items are actually capturing concepts which are relevant and associated with change in treatment, thus assuring that the measure has high content validity. Effect-size statistics are used to assess responsiveness. An effect size is a measure of the strength of the relationship between two variables in a statistical population and can convey whether an observed difference is substantively important. This is in contrast to a statistical significance test, which assesses whether a relationship could be due to chance, regardless of the strength of the apparent relationship in the data. Changes in patients’ condition was measured both by the patients themselves and by physician's assessment of the patient.

The IMPROVE study, was a multi-centre, open-labelled, non-randomised and non-interventional longitudinal observational study of the safety and efficacy of NovoMix 30 (biphasic insulin aspart 30) for the treatment of diabetes in routine practice with type 2 diabetes⁹. The DiabMedSat (Diabetes Medication Satisfaction Measure) was included in the IMPROVE study to examine the impact of treatment on patient reported TS. The DiabMedSat is a disease-specific PRO measure of TS designed to assess TS in persons with both type 1 and type 2 disease and across multiple treatment modalities (oral, syringe, pen)¹⁰ (Appendix 1 and available for research and clinical use at www.PROQOLID.org). The DiabMedSat has 22 items, assessing three TS domains of Efficacy, Treatment Burden and Symptoms (Side Effects) as well as overall TS and has been shown to be a valid and reliable measure^{11, 12}. Higher domain scores indicate greater TS for the concept measured by that domain. This study continues the iterative process of PRO measure validation by examining the responsiveness of the DiabMedSat.

Patients and methods

The responsiveness analyses used data collected in the IMPROVE study, a multi-centre, multinational, 26-week, open-labelled, non-randomised and non-interventional observational study of the safety and efficacy of biphasic insulin aspart 30 for the treatment of diabetes in routine practice patients with type 2 diabetes including newly-diagnosed patients who have never received insulin. The IMPROVE study included 52,419 patients with type 2 diabetes. After 26 weeks, 53% of patients experienced a significant improvement in glycaemic control (HbA_1c <7.0%). The global cohort also experienced a significant improvement in fasting blood sugar (4.3 mmol/L reduction, p<0.0001) and a significant improvement in body weight (0.1 kg reduction, p<0.0001)¹³.

Responsiveness analyses were conducted with all patients who had both a baseline and week 26 (end of study treatment) DiabMedSat score. Patients who at baseline were not treated with an anti-diabetic agent (e.g., on diet/exercise) were excluded as the DiabMedSat questionnaire is not relevant in this group. All statistical tests were two-tailed and conducted with an alpha level of 0.05 as minimal threshold for significance. As the DiabMedSat is intended to be used as either a total score or as independent domains, the responsiveness testing was conducted for both the total and domain scores.

Responsiveness of the DiabMedSat was assessed according to an a priori analysis plan to examine both ‘internal’ and ‘external’ responsiveness. Internal responsiveness refers to a measure's ability to change over a pre-specified timeframe whereas external responsiveness is the extent to which a measure's degree of change corresponds to a reference value or measure⁸.

To examine internal responsiveness, t-tests were used to examine differences in DiabMedSat scores between baseline and week 26 values. The effect size for this change was also examined by calculating Cohen's d (the mean change in score divided by the standard deviation of the mean baseline score). According to Cohen, an effect size of 0.2–0.3 was considered a small effect, around 0.5 (0.4–0.7) a medium effect and 0.8 or above a large effect¹⁴. Additionally, t-tests for change in scores for each individual DiabMedSat item and by country were also examined.

External responsiveness was examined by testing the ability of the DiabMedSat to differ between patients who: (1) met the HbA_1c goal, improved but did not meet goal, or showed no improvement at week 26, and (2) experienced minor hypoglycaemic episodes (during the past year) versus those that did not.

Finally, to begin to understand how to interpret change in scores, the minimally important difference (MID) was examined for the relationship between DiabMedSat scores and patient global rating of satisfaction with their diabetes medication. The patient global rating scale used a 7-point response scale (extremely dissatisfied, very dissatisfied, slightly dissatisfied, neither satisfied or dissatisfied, slightly satisfied, very satisfied, extremely satisfied) for the question, ‘Overall, thinking about each of the aspects of your diabetes medication(s), how dissatisfied or satisfied have you been with your current diabetes medication(s)?’ The difference between ‘neither dissatisfied or satisfied’ and ‘slightly satisfied’ was examined for the MID. Additionally, the MID was also examined according to physician-rated satisfaction with patients’ HbA_1c level at week 26. The physician global rating scale used a 5-point response scale (very satisfied, satisfied, neutral, dissatisfied, very dissatisfied) to answer the question, ‘Considering the HbA_1c target that you have set for your patients, how satisfied are you with NovoMix30 achieving the targeted HbA_1c?’ The difference between ‘neutral’ and ‘satisfied’ was examined for the MID. The MID was calculated by examining the mean and standard deviation of the identified response options for the global measure. Standardised response means (SRM) were calculated for changes within satisfaction categories by dividing the mean change by the standard deviation of the change scores¹⁵. An SRM >0.8 is considered large.

Results

Sample description

A total of 18,817 patients from Canada, China, India, Italy, Japan, Poland and Russia who participated in the IMPROVE study had both baseline and week 26 DiabMedSat scores were included in the analyses. Subjects had a mean age of 57.6 years, having had diabetes for an average of 7.4 years and a most recent baseline HbA_1c of 9.2 (Table 1).

Table 1. Sample description.

Age	Mean (SD) (n=18,813)	57.6 (12.0)
Gender	n (%) Male	10,509 (55.8%)
	n (%) Female	8,308 (44.3%)
Body mass index (BMI)	Mean (SD) Baseline (n=18,809)	26.6 (5.1)
HbA1c	Mean (SD) Most recent (n=14,098)	9.2 (1.9)
Diabetes duration (years)	Mean (SD) (n=18,793)	7.4 (5.8)
No. of diabetes-related complications	Mean (SD) Baseline (n=11,579)	2.0 (1.1)
Pre-study therapy	n (%) OAD (oral anti-diabetics) only	15,841 (84.2%)
	n (%) Insulin only	962 (5.1%)
	n (%) Insulin + OAD	2,014 (10.7%)
Reasons for starting biphasic insulin aspart 30	n (%) Easy start of therapy	11,625 (61.8%)
	n (%) Easy identification of insulin therapy	9,283 (49.3%)
	n (%) To improve HbA1c	15,008 (79.8%)
	n (%) To improve FBG	15,109 (80.3%)
	n (%) To improve PPBG	14,431 (76.7%)
	n (%) Reduce risk of hypoglycaemia	7,749 (41.2%)
	n (%) Patient dissatisfaction with previous therapy	9,682 (51.5%)
	n (%) Side effects from previous therapy	3,874 (20.6%)
	n (%) Change due to insulin pen	1,501 (8.0%)
	n (%) Allow for mealtime administration	8,021 (42.6%)

*Not mutually exclusive categories.

FPG, fasting blood glucose; PPBG, postprandial blood glucose.

Internal responsiveness

As shown in Table 2, the DiabMedSat total score as well as each domain changed significantly after 26 weeks of treatment with biphasic insulin aspart 30. Additionally, the effect size for these changes ranged from 0.54 to 1.65 indicating that the ability of the DiabMedSat to detect change was moderate-to-large in size. Mean score changes ranged from 11 (Symptoms (Side Effect) domain) to 28 (Efficacy domain) with the total score changing approximately 17 points (Table 2).

Table 2. DiabMedSat change with treatment.

DiabMedSat	n	Baseline Mean (SD)	Week 26 Mean (SD)	Change score Mean (SD)	t-stat (sig.)	Effect size
Efficacy	18,376	42.9 (17.2)	71.2 (15.3)	28.3 (22.3)	172.0 (p<0.001)	1.645
Symptoms	18,423	64.4 (20.0)	75.1 (15.6)	10.7 (20.8)	70.1 (p<0.001)	0.535
Burden	18,090	63.6 (14.7)	76.0 (14.2)	12.4 (18.2)	90.8 (p<0.001)	0.844
Total	18,761	56.9 (13.3)	74.0 (11.9)	17.1 (16.1)	145.3 (p<0.001)	1.286

Effect size = mean change in score divided by the standard deviation of mean baseline score.

Closer examination of the DiabMedSat found that each item was sensitive to changes (showed significant improvement) in the study (Table 3).

Table 3. Responsiveness of individual DiabMedSat items.

DiabMedSat	Baseline Mean (SD)	Week 26 Mean (SD)	t-stat (sig.)
Burden
The ease & convenience	4.8 (1.1)	2.9 (1.0)	123.3 (p<0.001)
How the medications interfere with your daily life	2.7 (1.0)	1.9 (0.9)	66.5 (p<0.001)
The need to adjust the dosing	2.5 (1.0)	1.7 (0.9)	59.8 (p<0.001)
Number of times you need to take your medications	2.5 (0.9)	1.8 (0.8)	68.5 (p<0.001)
Amount of home monitoring	2.4 (1.0)	2.0 (0.9)	53.3 (p<0.001)
Be flexible with planning meals	2.9 (1.0)	2.5 (1.1)	29.3 (p<0.001)
Follow your recommended diet	2.8 (0.9)	2.2 (1.1)	43.7 (p<0.001)
How much burden to take medications	2.4 (0.9)	1.9 (0.9)	55.8 (p<0.001)
How embarrassed you felt	2.3 (0.9)	1.8 (0.9)	41.6 (p<0.001)
How difficult to plan	2.3 (0.9)	1.9 (0.9)	38.3 (p<0.001)
Do your recommended physical activity	2.7 (0.9)	2.4 (1.0)	43.1 (p<0.001)
Efficacy
Keep your blood sugar stable	4.7 (1.1)	2.8 (1.0)	128.5 (p<0.001)
Impact on physical well-being	4.6 (1.2)	2.8 (1.0)	133.3 (p<0.001)
Help you from feeling tired	4.5 (1.0)	2.8 (0.9)	131.2 (p<0.001)
Impact on emotional well-being	4.6 (1.1)	2.7 (1.0)	139.9 (p<0.001)
How worried med not prevent complication	2.6 (0.9)	1.8 (0.9)	74.8 (p<0.001)
Symptoms
Gas and bloating	2.9 (1.6)	2.7 (2.1)	12.9 (p<0.001)
Low blood sugar	3.0 (1.7)	2.5 (1.8)	28.3 (p<0.001)
Diarrhoea	2.9 (1.7)	2.7 (2.0)	11.5 (p<0.001)
Unwanted weight gain	2.8 (1.5)	2.2 (1.5)	25.8 (p<0.001)
Pain or discomfort	2.8 (1.8)	2.3 (1.7)	26.7 (p<0.001)

External responsiveness

To assess external responsiveness, change in DiabMedSat scores from baseline to week 26 were examined by category of HbA_1c goal attainment set at baseline for each patient. A significant difference was found for the total and each domain score comparing those patients who met their HbA_1c goal set at baseline, those who improved but did not meet goal and those who did not improve with treatment. A linear relationship between categories of change was found such that the ‘met goal’ group mean score was the highest, the ‘improved but did not meet goal’ mean score was slightly less and the ‘no improvement’ group mean score was the lowest at week 26. The largest difference in mean scores between the ‘met goal’ group and the ‘no improvement’ group was seen in the Efficacy domain. These differences were controlled for age, baseline body mass index and duration of diabetes.

When examining the change in DiabMedSat scores from baseline to week 26 comparing those who did or did not experience minor hypoglycaemic episodes during the study, a significant difference was also seen between the two groups (Tables 4 and 5).

Table 4. DiabMedSat scores by HbA_1c categories of change at week 26.

DiabMedSat	Met goal (n=6,095)	Improved but did not meet goal (n=11,999)	No improvement (n=723)	p-value (sig.)
Efficacy	73.8 (14.1)	70.3 (15.5)	64.3 (17.6)	181.1 (p<0.001)
Symptoms	75.4 (15.1)	75.1 (15.9)	73.2 (15.9)	6.1 (p<0.01)
Burden	77.2 (13.7)	75.2 (14.5)	76.1 (14.9)	38.8 (p<0.001)
Total	75.4 (11.3)	73.5 (12.1)	71.2 (12.4)	75.3 (p<0.001)

Table 5. DiabMedSat scores by experience of minor hypoglycaemic episodes categories at week 26.

DiabMedSat	No hypoglycaemic events (n=16,396)	One or more hypoglycaemic events (n=2,355)	p-value (sig.)
Efficacy	71.4 (15.4)	69.1 (15.3)	28.7 (p<0.001)
Symptoms	75.3 (15.8)	71.9 (13.4)	62.4 (p<0.001)
Burden	76.1 (14.3)	74.0 (13.9)	12.6 (p<0.001)
Total	74.2 (12.0)	71.8 (11.0)	53.4 (p<0.001)

Scores for both patient overall satisfaction with their medication at week 26 and physician global satisfaction scores for improvement in HbA_1c were also examined. As seen in Table 6, mean score differences from baseline to week 26 for the total score and each domain, were higher for the group indicating ‘Slightly Satisfied’ than the group indicating ‘Neither Dissatisfied nor Satisfied’ on the patient global rating of satisfaction with their diabetes medication. The results were similar using the physician-rated satisfaction with patients’ HbA_1c level at week 26 where mean differences were larger in the group indicating ‘Satisfied’ than the group indicating ‘Neutral’. SRMs were also larger in the satisfied response groups. MIDs ranged from 5.3 to 11.7 for the patient global rating and 5.3 to 10.2 for the physician rating. Although the patient and physician MID were similar, the largest difference between physicians and patients was found for the Burden domain and the smallest difference was found for the total score (Table 6).

Table 6. DiabMedSat ability to detect change.

DiabMedSat domain	Patient global rating of satisfaction with their diabetes medication
	Mean difference between baseline and week 26				MID (Change between groups)
	Neither dissatisfied nor satisfied	n	Slightly satisfied	n
Efficacy	9.6 (17.5)	1,498	21.3 (18.3)	5,004	11.7
SRM	0.55		1.16
Symptoms	3.7 (20.2)	1,499	9.0 (21.1)	5,012	5.3
SRM	0.18		0.43
Burden	2.4 (16.0)	1,482	7.7 (16.0)	4,968	5.3
SRM	0.15		0.48
Total	5.3 (13.2)	1,526	12.6 (14.2)	5,101	7.3
SRM	0.40		0.89
	Physician-rated satisfaction with patients’ HbA1c level at week 26
	Mean difference between baseline and week 26
	Neutral	n	Satisfied	n	Change score
Efficacy	18.1 (21.8)	1,904	28.3 (21.2)	11,193	10.2
SRM	0.83		1.34
Symptoms	5.6 (19.2)	1,913	10.9 (20.7)	11,205	5.3
SRM	0.29		0.53
Burden	5.8 (17.8)	1,872	12.4 (17.6)	11,012	6.6
SRM	0.33		0.70
Total	9.8 (15.1)	1,951	17.2 (15.5)	11,507	7.4
SRM	0.65		1.11

SRM = standardised response mean (mean score change divided by the standard deviation of the change score)¹⁵.

Discussion

The study clearly demonstrated that the DiabMedSat is able to adequately capture change in TS for patients who respond to treatment. Given the effect sizes found for this significant change from baseline to end of treatment, the DiabMedSat total score as well as each domain score can be considered highly responsive. Further, the sample from the study was culturally very diverse with countries spanning continents and encompassing major differences in lifestyle, structure of healthcare system, payer issues and healthcare delivery issues. Despite these differences, the DiabMedSat was found to be responsive in all countries suggesting that the measure is not culturally sensitive and is appropriate for use in a wide spectrum of cultures and countries.

In addition to the patient MID, a physician MID was also examined. These global items were phrased differently for the physician versus patient to capture the most relevant and appropriate outcomes for the person answering the question. Thus the physician global question asked only about efficacy as related to HbA_1c and the patient global item asked about their total treatment experience. A limitation to this analysis was that there was a difference between the patient and physician MID response scales. The physician global rating scale used a 5-point response scale (very satisfied, satisfied, neutral, dissatisfied, very dissatisfied) whereas the patient global rating scale used a 7-point response scale (extremely dissatisfied, very dissatisfied, slightly dissatisfied, neither satisfied or dissatisfied, slightly satisfied, very satisfied, extremely satisfied). Unfortunately this difference makes it difficult to compare across physician and patient responses when interpreting the MID.

Every item was responsive to treatment, further supporting the content validity of the measure in that it appears that every item measures a concept of importance to this population. However, examination of the content of the items for the Symptom domain suggests that the domain is inappropriately named. The developers intended the domain to assess symptoms of treatment rather than symptoms of the disease. However, the authors now realize that this may be confusing even though the items in the domain clearly identify side-effects of diabetes treatments. Therefore, to more clearly reflect the item content, the name of this domain has been changed from Symptoms to Side Effects. This change in domain name does not reflect any change in the items within the domain or the psychometric properties previously examined¹⁰. Additionally, as the domain names are not provided on the written version of the measure, this change does not influence the perception of item meaning or choice of response options.

The data for these analyses was collected in an observational study rather than in a clinical trial design which examines treatment impact under highly structured and prescribed conditions and most often includes a control group. Thus, placebo effects should be considered when interpreting observational study findings. Observational data does not replace clinical trial data, but rather offer additional effectiveness information to support effectiveness findings. Thus these analyses which have examined responsiveness under real world conditions in primary and secondary care settings may have greater relevance for patients and clinicians in usual care practice than clinical trial data.

Conclusion

Given that the DiabMedSat has been found to be highly responsive, clinicians may consider using it as a disease management tool to assess TS with both new and ongoing diabetes medications. Improving understanding of patient reported burden and side-effects as well as efficacy should allow clinicians to better design and tailor disease management programs for their patients and then monitor the impact of these efforts over time. Identifying programs and treatments which improve TS should in turn result in improved compliance, health outcomes and improved health related quality of life.

Transparency

Declaration of funding: This study was funded by Novo Nordisk.

Declaration of financial/other relationships: M.B. and D.M.B. have disclosed that they are advisors/paid consultants to Novo Nordisk. T.C. and J.H.K. have disclosed that they are employees of Novo Nordisk. All authors participated in the conceptualisation, data interpretation and manuscript preparation.

The peer reviewers of this paper have disclosed that they have no relevant financial relationships.

Acknowledgment:

The authors have disclosed that they had no outside editorial assistance in preparing this manuscript.

Notes

* NovoMix 30 is a registered trademark of Novo Nordisk A/S, Bagsværd, Denmark.

* NovoMix 30 is a registered trademark of Novo Nordisk A/S, Bagsværd, Denmark.

Appendix

The DiabMedSat

Your diabetes treatment is a combination of diet, exercise and medications.

The following questions are concerned only with the MEDICATIONS (pills and/or insulin) that you take for your diabetes. If you take medications for other conditions, please try to think only about the medications you take for diabetes when answering the questions.

•	If you take more than one medication for your diabetes, please consider all of your diabetes medications when answering these questions.
•	Please check the box that most closely represents how you have felt about your diabetes medications over the PAST 2 WEEKS. Please check only one box for each question.
•	Remember there are no right or wrong answers to these questions.

Over the PAST 2 WEEKS, how BOTHERED have you been by:

Over the PAST 2 WEEKS, have you been bothered by any of the following due to your diabetes medication?

Over the past 2 weeks, how DISSATISFIED or SATISFIED have you been with your diabetes medication(s) ability to:

Overall, over the past 2 weeks, how DISSATISFIED or SATISFIED have you been with:

Thinking about your diabetes medication(s) over the past 2 weeks:

Over the past 2 weeks, how often has taking your diabetes medication(s) as prescribed INTERFERED WITH your ability to:

Overall, thinking about each of the aspects of your diabetes medication(s) mentioned above, how DISSATISFIED or SATISFIED have you been with your current diabetes medication(s)?