A cultural adaptation and validation study of a self-report measure of the extent of and reasons for medication nonadherence among patients with diabetes in Singapore

Abstract

Background

This self-report measure is a new instrument to measure the extent of and reasons for medication adherence separately. However, few studies have assessed its psychometric properties in diabetic patients and also in Asian populations.

Objectives

To validate this self-report measure in diabetic patients in Singapore.

Methods

We collected data prospectively using a questionnaire among 393 diabetic patients from hospitals in Singapore from July 2018 to January 2019. Using the COnsensus-based Standards for the selection of health Measurement INstruments framework, we assessed face validity, internal consistency, test–retest reliability, structural validity, and measurement error. We tested four a priori hypotheses on correlation of extent score with patient-reported outcome measures to assess construct validity. We examined cross-cultural validity via measurement invariance across gender, age groups, and languages.

Results

We performed cognitive interviews with 30 consenting English-literate, Chinese-literate, and Malay-literate (10 patients per language) diabetic patients (age range 48–76 years, 53% male, disease duration range 1–30 years) and face validity was supported. Among 393 patients (mean age: 59.4±12.2 years, 50.9% female, 52.4% Chinese), we showed moderate internal consistency (Cronbach’s alpha =0.67) and test–retest reliability (intra-class coefficient=0.56 [95% CI 0.37–0.70]). We calculated smallest detectable change as 0.80. We established construct validity by meeting all four hypotheses. We showed structural validity as confirmatory factor analysis confirmed a one-factor model, with excellent fit statistics (Comparative Fit Index=1.0; Tucker-Lewis Index=1.0; Root Mean Square Error of Approximation<0.001; Standardized Root Mean Residuals<0.001). Analysis of cross-cultural validity supported configural invariance model but not metric invariance and scalar invariance model. Caution must be taken against directly comparing extent scores across gender, age groups, and languages.

Conclusion

This self-report measure is valid and reliable in measuring medication adherence in diabetic patients in Singapore.

Keywords:

• adherence
• patient-reported outcome
• quality of life
• psychometric
• Singapore
• diabetes

Introduction

Diabetes mellitus is a metabolic disorder characterized by resistance to insulin action, insufficient insulin secretion, or both,¹ causing hyperglycemia, which is an increase in blood glucose level. It is predicted that the number of people with diabetes worldwide will increase from 382 million people in 2013 to 592 million by 2035.²

Medication adherence is necessary for improved health outcomes and lower health care costs, especially in chronic diseases like diabetes.³ Conversely, poor adherence in chronic diseases leads to worsening health outcomes,^4,5 higher risk of hospitalization⁶ and higher mortality rate.⁷ Diabetic treatment regimens are complex, causing adherence to diabetic medications to be difficult⁸ and worse compared to other chronic diseases.⁹ Poor adherence to diabetic medications has been found to be related to poorer glycemic control¹⁰ and greater risk of diabetic complications,⁷ such as retinopathy, neuropathy, nephropathy, and cardiovascular disease. Improving medication adherence can be a more efficient strategy than changing or adding additional treatments.¹¹

Nonadherence can be measured directly and indirectly. Direct methods include measurement of concentrations of drugs or their metabolites or markers but are used infrequently due to difficulty, cost, and inability to provide feedback at the point of care.¹² Indirect methods are more frequently reported in literature than direct measures¹² and include pill counts, electronic monitoring devices, pharmacy refill records, and self-reports. Although there is no “gold” standard for measuring nonadherence,¹³ self-reports have unique advantages over other methods such as ease of completion, inexpensive, and being able to provide immediate feedback at the point of care.¹⁴

Given the high prevalence and health care cost associated with diabetes, a variety of behavioral interventions have been developed to increase adherence to diabetes medications.¹⁵ It is important to have a reliable, valid measure that assesses the extent of nonadherence and the reasons for nonadherence to identify patients who would benefit from intervention, inform the type of intervention to be done, and evaluate the effectiveness of intervention. There are many self-report measures of medication nonadherence validated in other patient populations like the 4-item and 8-item Morisky Medication Adherence Scale [MMAS-4 and MMAS-8],¹⁶ Medication Adherence Report Scale,¹⁷ and Hill-Bone Compliance Scale [Hill-Bone].¹⁸ However, studies have concluded that these current measures have inadequate reliability and validity across populations due to the design of these measures.¹⁹ Voils developed a self-report measure that seeks to address two issues with the existing self-report measures. First, existing measures conflate the extent of and reasons for medication nonadherence. The Voils measure resolves this issue by separately assessing the extent of and reasons for medication nonadherence using appropriate psychometric models.²⁰ Second, some measures are unclear when conceptualizing stability of nonadherence over time¹⁴ and thus the Voils measure sets a specific recall period of 7 days,¹⁴ which was validated through cognitive interviews. The measure has been used in randomized controlled trials to assess medication nonadherence in patients with hypertension,²¹ dyslipidemia,²² and Type 2 diabetes in America.²³

However, the self-report measure by Voils has not been validated in Singapore, and there are limited medication nonadherence self-reports validated in Singapore. Singapore is a multicultural society with three dominant ethnic groups – Chinese (74.2%), Malays (13.4%), and Indians (9.2%)²⁴ and there are significant proportions of residents (aged ≥15 years) who are literate only in one language (English: 10.6%, Chinese: 15.6%, Malay: 1.52%).²⁴ In order to apply the measure to patient populations in Singapore, it must be culturally relevant and translated to the appropriate languages. Assessing adherence to medications for diabetes in Singapore is important because the prevalence of diabetes is increasing. The number of people with diabetes in Singapore has increased from 2.0% of the population in 1975 to 12.7% in 2012²⁵ and is projected to rise to 22.7% in 2035.²⁶ Therefore, we aim to validate a self-report measure of the extent of and reasons for medication nonadherence by Voils in patients with diabetes in Singapore.

Methods

Study design

We conducted a two-phase study. The first phase involved cognitive interviews to assess cultural adaptation of the translated measures. The second phase involved a prospective cohort study, whereby all participants received the questionnaire, and then 2 weeks later a subset of those participants received the same questionnaire again. The goal of the first measurement component was to assess internal consistency and construct validity, whereas the goal of the repeated measure component was to assess the reliability of the extent of nonadherence scores across a short time period. We recruited patients with diabetes from multiple sites consisting of both public hospitals and primary care centers in Singapore from July 2018 to January 2019. Participants could choose English, Chinese, or Malay version of the questionnaire, depending on their language preference. The National Healthcare Group Domain Specific Review Board (Ref no.: 2018/00472) approved this study. All participants provided written informed consent.

Patients

We approached eligible participants for recruitment while they were in the waiting areas of the pharmacies. We included participants who were at least 21 years old, with diabetes treated with antidiabetic medications and on regular follow-up at these hospitals or primary care centers. We excluded participants who were not willing to provide informed consent.

The self-report measure

It is a self-administered, two-domain questionnaire that assesses the extent of and reasons for nonadherence over the past 7 days,¹⁴ with a Likert-type additive scale of five responses measuring frequency: none of the time, a little of the time, some of the time, most of the time, and every time.²² There are three items in the extent of nonadherence, which are: “I missed my medicine”, “I skipped a dose of my medication”, and “I did not take a dose of my medication”. The overall score was calculated by the average score of the three items. Higher scores indicate greater levels of nonadherence.²⁰ There are 18 items in the reasons for nonadherence which stand on their own as descriptors. The reasons scale is a causal indicator model¹⁴ and higher scores indicate greater endorsement of each reason for nonadherence.²⁰

Translation of the self-report measure

We followed the guidelines for translation and adaptation of self-report measures by the ISPOR Task Force for Translation and Cultural Adaptation.²⁷ Two researchers performed the forward translation of the English version of the self-report measure to Chinese and Malay. Another two independent researchers who were unaware of the original self-report measure performed the back translation. A panel of four multidisciplinary and bilingual clinical pharmacists with experiences with diabetic patients, which were different from the four researchers who translated the self-report measure, evaluated the three versions (the original, the forward translated, and the back translated). They ensured that the content, wording, and cognitive level of the Chinese and Malay versions were equivalent to the English version and that they had been appropriately adapted linguistically to Singaporeans.

Cognitive testing of the self-report measure

We conducted cognitive interviews with 30 consenting English-literate, Chinese-literate, and Malay-literate (ten participants per language) diabetic patients to evaluate the cultural relevance, comprehensiveness, scope, and acceptability of the self-report measure. We selected participants to represent a range of age, gender, and disease duration. The participants completed the questionnaire in the presence of a trained interviewer (YWL, CC, MHL) and the time of start and completion of the questionnaire were recorded. We answered any questions raised by the participants specific to the understanding of the questions by using a prescribed set of interpretation notes to validate the interpretation of the questions. We asked the participants about questions they had trouble answering and possible rephrasing of the questions. We also asked the participants on the ease of understanding, ambiguity, comprehensiveness, inappropriateness, and relevance of the questions for medication nonadherence.

Prospective cohort study

For the cross-sectional study, we asked participants who had provided written consent to complete a questionnaire, which consisted of demographic characteristics, clinical information, and patient-reported outcome measures (PROMs). Demographic characteristics included age, gender, ethnicity, and highest education level attained. We obtained clinical data from medical records, including disease duration and HbA1c in the past 3 months. PROMs included the self-report adherence measure being validated,²⁰ Culig Adherence Scale (CAS),²⁸ Medication Adherence Visual Analogue Scale (MAVAS),²⁹ and EuroQoL-5 Dimensions-5 Levels (EQ-5D-5L).³⁰

For the longitudinal component, we assessed medication adherence at two time points. We conducted face-to-face interviews at the first time point, which was at baseline (Day 0). We then performed telephone follow-up on Day 14, when the interviewer asked the same questions in the same questionnaire used at the face-to-face interviews for test–retest reliability. The purpose of the 14-day interval was to verify the consistency of adherence scores recorded at baseline.³¹ We chose this time period as it was recommended by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) framework to be sufficiently long enough to minimize recall bias while sufficiently short enough to fulfill the assumption of no significant change in medication adherence.³²

Comparison measures

CAS is also a two-part questionnaire that measures the extent of and reasons for nonadherence²⁸ separately with an indefinite duration of nonadherence. There is one item in the extent of nonadherence that asks, “When was the last time when you failed to take your medication?”, with six responses: last week, 1–2 weeks ago, 3–4 weeks ago, 1–3 months ago, more than 3 months ago and I never fail to take my medication on time. Higher scores indicate lower levels of nonadherence. There are 16 items in the reasons for nonadherence with a Likert-type additive scale of four responses: never, very rare (1–2 times a year), sometimes (3–5 times a year), and often (more than 5 times a year). Higher scores indicate greater occurrence of each reason for nonadherence. We chose CAS as it also separates extent and reasons of nonadherence and so is a good comparison with the self-report measure.

MAVAS measures the extent of nonadherence²⁹ over the past 6 months. It is a one-item measure that asks, “What percent of time over the past 6 months did you take your prescribed medication?”, with a continuous scale from 0% (not adherent) to 100% (fully adherent). Higher scores indicate greater levels of nonadherence. We chose MAVAS as ordinal scales have shown to have higher sensitivity and reproducibility compared to discrete scales.³³ It also permits easier administration and we can see changes in the measurement of nonadherence between continuous and ordinal scales.

EQ-5D-5L (ranging from − 0.5 to 1.0) is a generic health index measuring health utility³⁰ on the day of questionnaire administration. Health utility is computed from five dimensions of health: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. It includes a visual analog scale which records patients’ self-rated health status on a graduated (0–100) scale. Higher scores in both scales show higher health-related quality of life (HRQoL).³⁴ We chose EQ-5D-5L as it is well validated with measurement invariance across the English, Chinese, and Malay versions in Singapore.³⁵

Statistical analyses

We performed all statistical analyses using STATA SE 14.0 for Windows (StataCorp, College Station, TX, USA) and followed the COSMIN framework. We used the Shapiro–Wilk test to investigate the normality of the distribution of continuous variables. Due to normal distribution of the variables, we tabulated descriptive statistics as mean (SD) and categorical variables as n (%).

As the self-report measure contains two related but distinct nonadherence constructs, each construct must be assessed by a different type of psychometric model.²⁰ The extent of nonadherence is represented by an effect indicator model, where the common underlying latent variable (in this case level of nonadherence) determines the item responses.³⁶

Reliability is defined as consistency of the measure, which is the degree to which the measurement is free from measurement error.³⁷ We assessed internal consistency, the degree of the interrelatedness among the items,³⁷ using Cronbach’s alpha, where values of 0.6–0.7 show acceptable level of reliability,³⁸ 0.7–0.8 shows satisfactory level,³⁹ and ≥0.8 shows excellent level,³⁸ assuming all items were similar and measured a single construct. We assessed test–retest reliability via intraclass correlation coefficient (ICC) (two-way mixed effects model, single measure), where based on the 95% confidence interval, values of 0.50–0.75, 0.75–0.90, and >0.90 show moderate, good, and excellent reliability, respectively.⁴⁰ We calculated ICC for each extent item and the extent average score with time as fixed parameter and patients as random variable,⁴¹ assuming the construct was stable over the two time points. We assessed measurement error, the random error of a patient’s score that was not attributed to true changes in the construct,³⁷ by analyzing the smallest detectable change (SDC) based on the 95% limits of agreement using the formula: SDC=1.96×Standard Error of Measurement in mean difference in the extent score of the two assessments in the test–retest reliability sample×√2. It is sufficiently low when SDC<Minimal Important Change,³² which is the minimum change in average extent score that is important to patients.³⁷

We tested construct validity, the degree to which the extent scores were consistent with the hypotheses,³⁷ using four a priori hypotheses:

• 1) The self-report measure is moderately negatively correlated with CAS and MAVAS^28,29

• 2) The self-report measure is weakly positively correlated with HbA1c⁴²

• 3) The self-report measure is weakly negatively correlated with EQ-5D-5L⁴³

We assessed the correlation between the self-report measure and other PROMs via Spearman’s rank correlation. The linear relationships were considered negligible if the magnitude of the correlation coefficient is <=0.3, weak if >0.3 and ≤0.5, moderate if >0.5 and ≤0.7, high if >0.7 and ≤0.9, and very high if >0.9.⁴⁴ The self-report measure should correlate with stronger magnitude with instruments that measure similar domains.³² At least 75% of the results should be in accordance with the hypotheses to demonstrate construct validity.³² To reduce the likelihood of Type I error, we considered p-values <0.0125 as statistically significant after applying Bonferroni’s correction.

We assessed structural validity, the degree to which extent scores are an adequate reflection of the dimensionality of the construct,³⁷ using confirmatory factor analysis to confirm the goodness of fit of the self-report measure in relation to the construct, using the maximum-likelihood method.⁴⁵ We assessed the increment model fit by Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI), values >0.95 were indicative as good model fit.⁴⁶ We assessed the absolute model fit by Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Residuals (SRMR), RMSEA <0.06 and SRMR <0.08 were indicative of good fit.⁴⁶

We assessed cross-cultural validity because the self report measure was used in different “cultural” population in the study,⁴⁶ by examining measurement invariance, the statistical property of a measurement that indicates that the same underlying construct is being measured across groups or across time,⁴⁷ of extent scores of the self-report measure across age, gender, and language. As the median age of patients in the study was 60, we dichotomized the patients into <60 years old and ≥60 years old for age group. We created three language groups based on language of the questionnaire done: 1) English (n=199); 2) Chinese (n=112); and 3) Malay (n=94). We then conducted a multi-group confirmatory factor analysis by constructing the following three increasingly restrictive models step-wise: where all parameters were free (configural invariance), where factor loadings were invariant (metric invariance), where factor loadings and item intercepts were invariant (scalar invariance). For adequate sample size (total N>300) and sample sizes similar across groups, measurement invariance is seen if ∆CFI and ∆TLI ≤0.01, ∆RMSEA ≤0.15, ∆SRMR ≤0.03 (for metric invariance), and ∆SRMR ≤0.01 (for scalar invariance).⁴⁸

Reasons for nonadherence are represented by a causal indicator model, where each reason for nonadherence stands alone as a descriptive indicator because they may not be correlated.³⁶ We considered any participants who scored ≥2 on any extent item as nonadherent, as seen in other validations of this scale and other investigations of the longitudinal predictive validity of this scale.²² We recorded descriptive statistics for the reasons items. We did not calculate Cronbach’s alpha because it is inappropriate for causal indicators.⁴⁹

Results

Phase I: cognitive testing of the self-report measure

We conducted cognitive interviews with 30 consenting participants with diabetes (age range 48–76 years, 53% male, disease duration range 1–30 years). The participants took an average of 4 minutes to complete the self-report measure. In the English version, seven participants pointed out that the phrasing of the extent of nonadherence questions was vague and five participants could not understand the difference between Item 2 (I skipped a dose of my medicine) and Item 3 (I did not take a dose of my medicine). In the Chinese version, two participants felt that the instructions needed clarification, as “please think about your pills only (not interferon)” was inadvertently left (based on the version validated in patients with hepatitis C infection). Three participants felt that inclusion of the stem “I missed my medication because … ” for the reasons for nonadherence items was unnecessary, so it was dropped. No participants identified problems with the wording of the extent of nonadherence items or response scale. For the Malay version, no issues were identified with the instructions, items, or response scales. The revised Chinese and Malay versions were tested in a second round of cognitive interviews.

Phase II: prospective cohort study

For the cross-sectional study, we recruited 393 participants who are available for analysis as shown in Table 1. The participants were equally distributed between the two genders (50.9% female). The mean age of the participants was 59.4±12.2 years. Most were Chinese (52.4%), married (72.6%) and spoke English (77.6%), and less than half completed secondary education (35.1%). The mean HbA1c obtained from the medical records was 8.3±1.6%.

Table 1 Sociodemographic and clinical characteristics of patients

Sociodemographic characteristics of patients (N=393)
Age, Mean (SD)	59.3 (12.2)
Race, N (%)
Chinese	206 (52.4)
Malay	114 (29.0)
Tamil	60 (15.3)
Others	13 (3.3)
Gender, N (%)
Male	193 (49.1)
Female	200 (50.9)
Education, N (%)
No formal education	24 (6.1)
Primary education	63 (16.0)
Secondary education	138 (35.1)
Tertiary education	121 (30.8)
Others	47 (12.0)
Living arrangement, N (%) (n=392)
Living by myself	35 (8.9)
Living with my spouse only	125 (31.9)
Living with my spouse and children	137 (35.0)
Living with my children but not my spouse	44 (11.2)
Living with others who are not my spouse or children	51 (13.0)
Marital status, N (%) (n=391)
Single	45 (11.5)
Married	284 (72.6)
Divorced/separated/widowed	62 (15.9)
Years with Diabetes, N (%) (n=392)
≤10	170 (43.3)
11–20	114 (29.1)
21–30	78 (19.9)
31–40	23 (5.9)
≥41	5 (1.3)
Clinical characteristics of participants, mean (SD)
Baseline HbA1c (n=356)	8.3 (1.6)

Abbreviation: HbA1c, glycated hemoglobin.

Extent of nonadherence

Interpretability

Figure 1 shows the score distribution of the average score of the extent of nonadherence in the self-report measure from the 393 participants. The extent score had a median of 1.3, with a range of 1–4. Most patients were adherent to their diabetic medications the past 7 days, as shown by the large proportion of patients scoring 1 (61.1%). Due to the small number of nonadherent patients, we dichotomized the scores to adherence (score of 1) and nonadherence (scores above 1). Table S1 shows descriptive statistics for the extent questions for all languages, English, Chinese, and Malay version.

Figure 1 The scores of the extent of nonadherence.

Notes: The scores are an average of the three extent items and ranged from 1 to 5. A score of 1 represents patients with complete adherence while a score of 5 represents patients with the poorest adherence over the past 7 days.

Reliability

The Cronbach’s alpha for all languages was 0.67 while the Cronbach’s alphas for English, Chinese, and Malay version were 0.65, 0.70, 0.73, respectively, demonstrating acceptable internal consistency. For the longitudinal component, we assessed 118 participants for test–retest reliability, which was moderate as ICCs of the individual extent items were 0.52, 0.54, and 0.52 for Item 1, 2, and 3, respectively, and the average score had an ICC of 0.56 (95% CI 0.37–0.70). We calculated the SDC as 0.80.

Construct validity

We fulfilled all four hypotheses about the magnitude and direction of correlation between the self-report measure and CAS, MAVAS, EQ-5D-5L, and HbA1c (Table 2) and all were statistically significant with p-values <0.0125, supporting construct validity. As expected, we observed stronger correlations with PROMs that measure adherence (ie, CAS and MAVAS) than those that measure other constructs, providing evidence of convergent and discriminant validity, respectively.

Table 2 Construct validity of the self-report measure (n=393)

Parameters compared with the self-report measure	A priori hypotheses		Spearman’s rank correlation coefficient	p-value	Hypotheses met
	Magnitude	Direction
CAS	Moderate	Negative	−0.609	<0.001	Yes
MAVAS	Moderate	Negative	−0.491	<0.001	Yes
EQ-5D-5L	Weak	Negative	−0.186	<0.001	Yes
HbA1c	Weak	Positive	0.132	0.012	Yes

Abbreviations: CAS, Culig Adherence Scale; MAVAS, Medication Adherence Visual Analogue Scale; EQ-5D-5L, EuroQol 5-Dimensional 5-Levels Questionnaire; HbA1c, glycated hemoglobin.

Structural validity

The one-factor model proposed for the confirmatory factor analysis of the self-report measure displayed the following fit indices (CFI=1.0; TLI=1.0; RMSEA<0.001; SRMR<0.001), which showed that the model yielded a good fit to the data, thus supporting structural validity.

Cross-cultural validity

The one-factor model proposed yielded great fit for age group, gender and language (CFI=1.000; TLI=1.000; RMSEA<0.001; SRMR<0.001 for all three). This showed that the single-factor structure of the self-report measure was equivalent across age group, gender, and language. However, the metric and scalar invariance model did not fit well according to descriptive fit indexes. ∆CFI, ∆TLI, ∆RMSEA, and ∆SRMR all did not meet the cutoff threshold (Table S2).

Reasons for nonadherence

Table 3 provides descriptive statistics for each reason of nonadherence across the 131 participants that reported nonadherence. Most participants reported forgetting as the most common reason (n=112, 73.2%), followed by being out of routine (n=92, 60.1%), being unable to meet the food requirements (n=76, 49.7%), not having medication with them (n=73, 47.7%), and being too late with their dose (n=73, 47.7%). The remaining reasons were on 13.1–36.6% of nonadherence occasions. On five nonadherence occasions (3.3%), the participants did not endorse any reasons for nonadherence despite them reporting nonadherence on the extent scale.

Table 3 Endorsement of reasons for nonadherence

Reasons for nonadherence (n=153)	Nonadherence occasions endorsed N (%)
I forgot	112 (73.2)
I was out of my routine	92 (60.1)
I could not meet the food requirements	76 (49.7)
I did not have my medicines with me	73 (47.7)
I was too late with my dose	73 (47.7)
I was asleep	56 (36.6)
I ran out of medication	43 (28.1)
I could not afford the medication	42 (27.5)
The medication caused side effects	38 (24.8)
I was feeling too sick to take it	37 (24.2)
I had other medications to take	31 (20.3)
The medication affected my sex life	27 (17.7)
Treatment was hard on my family	24 (15.7)
I was afraid the medication would interact with other medications I take	24 (15.7)
The medication was not working	23 (15.0)
I did not want others to see my medications	20 (13.1)
I could not get answers to my questions about the medication	20 (13.1)
There was no one to help me	18 (11.8)

Discussion

This study provides support for adequate face validity, internal consistency, test–retest reliability, construct validity, and structural validity of the self-report adherence measure in Singapore. To our knowledge, this is the first psychometric validation of the self-report measure in measuring the extent of and reasons for medication nonadherence in diabetic patients in a multilingual Asian city. This study is also the first to show the cross-cultural validity of a medication nonadherence questionnaire in Asia. Having a large sample size, following the COSMIN framework rigorously and formulating hypotheses a priori for construct validity were the key strengths of this study.

From the cognitive interviews, we made changes to the extent items of the English version, which were the addition of “by accident” to Item 1 and “on purpose” to Item 2, as participants reported that they could not understand the difference between both items. The multi-lingual Singaporeans that completed the self-report measure in this study as compared to English-speaking only Americans that Voils herself developed the original version from,²⁰ might account for differences in understanding of the English items, that compelled us to clarify the wording. The original version that Voils developed was done through several rounds of cognitive interviews with monolingual Americans²⁰ and thus, that version is still recommended to be used for Americans due to the extensive work and evidence of reliability and validity. In other multilingual countries, re-validation of the self-report measure is necessary for proper linguistic adaptation, to ensure the measure remains culturally relevant to the respective context. This is further supported with the higher Cronbach’s alpha in the Chinese and Malay versions as compared to the English version, which shows that the modification of the self-report measure was more appropriate for the Chinese and Malay versions than the English version.

For the Chinese version, the stem “I missed my medication because …” for the reasons for nonadherence items was dropped as the participants voiced out that it was unnecessary. Although the stem was included in the original version as Voils found in her cognitive interviews that patients started to respond about the reasons for nonadherence items in general rather than as reasons for missing a dose in the past 7 days,²⁰ the three participants during our cognitive interviews mentioned that they understood what was asked without needing the stem to be repeated for every reason.

Our reported adherence rate (61%) is slightly higher than that observed in patients taking blood pressure medications (40%)²⁰ and statins (57%).²² This may reflect that the actual level of adherence to medications in diabetic patients is higher than in other populations. Alternatively, this may reflect that our patient population was more likely to overestimate adherence due to social desirability bias, where patients give responses that followed the advice of their health care providers,⁵⁰ non-response bias, where nonadherent patients were less likely to participate in the study,⁵¹ and recall bias.⁵² These three types of bias are well-known limitations of self-report measures.

The test–retest reliability and structural validity of the extent of nonadherence were comparable to a study done by Voils et al.²⁰ However, the internal consistency was found to be lower (alpha =0.65, 0.70, and 0.73 for English, Chinese and Malay, respectively) compared to Voils’ study on hypertensive patients (alpha=0.84)²⁰ and 2014 (alpha=0.78–0.94 across four occasions),²¹ and Blalock’s 2019 study on dyslipidemia patients (alpha =0.90).²² For the English version, this may be due to the changes made to the extent items to clarify the meaning to participants. Because people may respond in one way to “by accident” and another way to “on purpose,” as there are two different aspects of medication nonadherence measured: unintentional (Item 1) and intentional (Item 2),⁵³ the inter-item correlations will be attenuated.⁵⁴ As Cronbach’s alpha measures both the homogeneity of the items and the homogeneity of what is being assessed (medication nonadherence),⁵⁵ internal consistency may decrease with increasing clarity of the extent questions as shown by the lower Cronbach’s alpha. This suggests a need to refine the items further in future research. The alphas for the Chinese and Malay versions were acceptable, even if somewhat lower than the original version. Cultural adaptations of measures do not always yield identical psychometric properties as the original measure.

Construct validity was supported in hypotheses testing, as there was fulfillment of all four a priori hypotheses in both magnitude and direction. We found a small, positive correlation between the self-report measure and HbA1c; this was expected given that adherence and glycemic control are different constructs, represent different time periods (7 days versus 3 months), and that variables other than adherence contribute to glycemic control (eg, dietary intake, genetics).⁴² We also found a small, negative correlation between the self-report measure and EQ-5D-5L, thus reflecting the fact that adherence and HRQoL are different constructs, both represent different time periods too (7 days versus day of administration), and that health status does not depend on adherence alone.⁵⁶

Analysis of cross-cultural validity supported the configural invariance model but not the metric invariance and scalar invariance models in the extent of nonadherence. Clinicians can still use the self-report measure in patients with varying gender, age groups, and languages, but caution must be taken against directly comparing the extent scores across these categories. Any direct comparison of the extent scores across languages should consider the fact that any observed differences could be due to measurement equivalence/invariance.

The reasons of nonadherence in the self-report measure were similar to a qualitative study by Shiyanbola et al, where they reported diabetic patients were non-compliant to antidiabetics due to forgetting, concerns about side effects, doubts about effectiveness, medications not physically available to them, caregivers unavailable to facilitate use of medicines, and lack of knowledge on how to ask providers questions regarding medications.⁵⁷ However, some reasons reported by Shiyanbola et al, such as fear of taking medicines, frustration, and tiredness of taking for long time and denial of having diabetes as the disease did not show any symptoms to patients,⁵⁷ were not reflected in the self-report measure. The reasons of nonadherence were also similar to a cross-sectional study in American veterans by Weidenbacher et al, which included “I forgot”, “I ran out of medication”, “I had other medications to take”, “The medication caused side effects”, “I had other medications to take”, “I was feeling too sick to take it”.²³ However, other reasons reported by Weidenbacher et al, such as reasons under the categories of “Negative expectations or worry” and “Does not take condition seriously”,²³ were not shown in the self-report measure. The five participants who did not indicate any reasons for nonadherence despite reporting nonadherence on the extent scale might have the other reasons as stated by Shiyanbola et al⁵⁷ and Weidenbacher et al,²³ and thus further studies can be done to improve the comprehensiveness of this self-report measure. These five participants may not have indicated any reasons for nonadherence because they had no specific reason. They might have known that they did not take their medications but did not know why they did not do so.

There are some limitations to this study. First, the convenience sampling procedure may result in selection bias as nonadherent patients were less likely to visit their physicians⁵⁸ and therefore less likely to participate in the study.⁵¹ However, the aim of this study was to assess the psychometric properties of the self-report measure, and the relationship between variables should hold even if the mean level of adherence was inflated, provided there was sufficient variability. Therefore, the impact of having a convenience sample will be minimal in influencing the study results. Second, we did not estimate Minimal Important Change due to the difficulty in finding a proven intervention that improves medication adherence. Thus, we only reported SDC, in line with recommendations from COSMIN.³⁷ Third, we did not assess criterion validity using an objective measure such as electronic drug monitoring or pill count,⁵⁹ and thus future studies can consider validating the self-report measure against more objective measures. Fourth, we did not specify if the participants took oral hypoglycemic agents or insulin because we wanted to validate the self-report measure for use across a wider diabetic population. Insulin adherence is complicated due to feasibility challenges associated with prescription claims data and injectable medication days’ supply,⁶⁰ and no quality measure is currently associated with adherence to insulin treatment.⁶⁰ Further studies can validate this self-report measure in measuring insulin adherence specifically. Fifth, there were five participants who did not indicate any reasons for nonadherence despite reporting nonadherence on the extent scale. This was due to us not including an open-ended option for participants to report any reasons that were not included in the list. Though upon asking the 30 participants whether there were any other reasons to add during the cognitive interviews, they did not mention any. This sample size is sufficient enough according to COSMIN guidelines³² and we have sampled a wide breadth of patients through purposive sampling. Future studies can include additional in-depth interviews or focus group discussions to elucidate many reasons for nonadherence from a larger sample of Asians. Last, we did not mention intentional and unintentional nonadherence because we wanted to cover the reasons for nonadherence as representatively in our target population as possible for the self-report measure to have content validity. Future studies can use the self-report measure to examine what proportion of medication nonadherence is intentional versus unintentional in our target population.

Conclusion

In conclusion, this study supports the face validity, internal consistency, test–retest reliability, construct validity and structural validity of the self-report measure to measure the extent of and reasons for medication nonadherence separately in diabetic patients in Singapore. This provides researchers and clinicians greater confidence to use the self-report measure to measure the extent of nonadherence in diabetic patients. Future interventions can be developed specifically for different reasons of nonadherence, and effectiveness of such interventions can be assessed with the self-report measure.

Acknowledgments

Full version of the self-report measure will be available on https://www.surgery.wisc.edu/research/researchers-labs/corrine-voils-phd/self-reported-medication-nonadherence-measurement/. The National Healthcare Group Pharmacy Transformation Office (NHGPTO) provided a non-restricted grant for this project. All patients provided written informed consent to participate in this research.

Disclosure

Dr Voils was supported by a Research Career Scientist award from the Department of Veterans Affairs Health Services Research & Development Service (RCS 14-443). The authors report no other conflicts of interest in this work.

Supplementary materials

Table S1 Proportion of extent of nonadherence

Items	Proportion of nonadherence N (%)
	All languages (n=393)	English (n=199)	Chinese (n=112)	Malay (n=101)
Item 1	119 (30.3)	77 (38.7)	24 (21.4)	20 (19.8)
Item 2	52 (13.2)	35 (17.6)	24 (21.4)	3 (3.0)
Item 3	69 (17.6)	42 (21.1)	11 (9.8)	18 (17.8)
Overall	153 (38.9)	93 (46.7)	35 (31.6)	27 (26.7)

Table S2 Cross-cultural validity of the self-report measure (n=393)

Model	Model comparison	BIC	RMSEA	∆RMSEA	CFI	∆CFI	TLI	∆TLI	SRMR	∆SRMR
Age: <60 y/o (n=185) vs ≥60 y/o (n=208)
M1		2191.137	0	-	1	-	1	-	0	-
M2	M2 vs M1	2185.163	0.123	0.123	0.948	0.052	0.896	0.104	0.078	0.078
M3	M3 vs M1	2184.180	0.139	0.139	0.867	0.133	0.867	0.133	0.067	0.067
Gender: Males (n=193) vs Females (n=200)
M1		2198.215	0	-	1	-	1	-	0.041	-
M2	M2 vs M1	2194.968	0.179	0.179	0.909	0.091	0.818	0.182	0.066	0.025
M3	M3 vs M1	2340.967	0.131	0.313	0.889	0.101	0.889	1.162	0.072	0.037
Language: English (n=199) vs Chinese (n=112) vs Malay (n=94)
M1		2134.305	0	-	1	-	1	-	0	-
M2	M2 vs M1	2148.769	0.237	0.237	0.788	0.212	0.682	0.318	0.148	0.148
M3	M3 vs M1	2151.214	0.221	0.221	0.633	0.367	0.725	0.275	0.121	0.121

Abbreviations: BIC, Bayesian Information Criterion; RMSEA, Root Mean Square Error of Approximation; CFI, Comparative Fit Index; TLI, Tucker-Lewis Index; SRMR Standardized Root Mean Residuals; M1, Configural Invariance Model; M2, Metric Invariance Model; M3, Scalar Invariance Model.