Construct validity and reproducibility of the Prescription Opioid Misuse And Abuse Questionnaire (POMAQ)

Abstract

Objective

The Prescription Opioid Misuse and Abuse Questionnaire was developed to identify prescription opioid abuse and misuse among patients with chronic pain, however, evidence of construct validity and reproducibility is needed.

Methods

Chronic pain patients were recruited from five Department of Defense Military Health System clinics across the United States. Construct validity was examined using subjective clinician-reported and patient-reported measures as well as objective information (e.g. hair/urine drug screens and electronic medical records). Test-retest reliability was assessed across 2 timepoints among a subgroup of patients with stable chronic pain.

Results

Of 3,263 screened patients, 938 (28.7%) met eligibility and were enrolled; 809 (86.2%) completed the Prescription Opioid Misuse and Abuse Questionnaire. Construct validity was supported by comparison to other validated questionnaires and hair and urine screens which yielded high agreements with patient reports on the Prescription Opioid Misuse and Abuse Questionnaire. Electronic medical record data supported patients’ Prescription Opioid Misuse and Abuse Questionnaire responses regarding physician and emergency room visits and opioid refills. The Prescription Opioid Misuse and Abuse Questionnaire had excellent test–retest reliability; the percentage agreement between the two Prescription Opioid Misuse and Abuse Questionnaire administrations was high (>90%) for most questions.

Discussion

Results suggest that the Prescription Opioid Misuse and Abuse Questionnaire is a valid and reproducible tool that can be used to assess the presence of prescription opioid misuse and abuse among patients with chronic pain.

Keywords:

• Construct validity
• Reproducibility
• Prescription Opioid Misuse and Abuse Questionnaire (POMAQ)
• patient reported outcome

Introduction

In 2016, over 20% (approximately 50 million) of the adult United States (US) population experienced chronic pain¹ with negative impacts on their physical and mental well-being, ability to function in the workplace, and ability to engage in social activities². Opioids are recommended for the treatment of acute and chronic pain and represent one of the most common classes of prescribed drugs in the US^3,4. Approximately 17% of the US population received one or more opioid prescriptions in 2017⁵. Inappropriate use of these medications, however, can lead to misuse and abuse, which poses serious risks to patients including opioid use disorder, overdose, and death^3,4. In 2017, misuse of prescription pain relievers was reported by over 11 million people (approximately 4% of individuals 12 years of age and older) in the US and slightly more than half (53.1%) of these individuals obtained their last prescription pain reliever from a friend or relative⁶. In response to the growing national trend in opioid-related overdose and deaths over the past decade, the US Department of Health and Humans Services declared a public health emergency in 2017, and the Centers for Disease Control and Prevention (CDC) published guidelines to clinicians for the appropriate prescribing of opioids for chronic pain^3,7.

In addition to reforming clinician prescribing practices to prevent opioid misuse and abuse, the US Food and Drug Administration (FDA) launched several initiatives to improve patient-focused research methods and instruments used to evaluate the safety of new or reformulated opioid pain medications⁸. In 2013, the FDA concluded that more data were needed regarding the risks associated with long-term use of extended-release/long-acting (ER/LA) opioids for the treatment of chronic pain⁹. As such, the FDA mandated that manufacturers of ER/LA opioids collaborate on a post-marketing requirement (PMR) to conduct a series of observational studies to evaluate the rates of treatment-emergent misuse, abuse, addiction, overdose, and death using validated measures. An objective of one of these observational studies was to develop and validate an instrument that would measure opioid-related misuse and abuse among patients with chronic pain who are prescribed long-term opioid therapy¹⁰. Though there are several patient screeners in use for assessing the risk of opioid abuse^11–15, none are currently validated for use in a chronic pain population or they are not designed to assess intentionality and/or do not distinguish between behaviors of misuse and abuse. The Self-Report Misuse, Abuse and Diversion of Prescription Opioids questionnaire (SR-MAD) was developed to assess patient behaviors related to opioid misuse and abuse ¹⁶ and was later extensively modified; questions were added to address the specific PMR research needs and capture all potential prescription opioid misuse and abuse behaviors as well as use and procurement of prescription and non-prescription opioids (and other illicit drugs). The modified questionnaire was renamed the Prescription Opioid Misuse and Abuse Questionnaire (POMAQ). Importantly, the basis for the POMAQ is to assess the intention of each behavior to ascertain if the patient-reported intent was associated with misuse or abuse behaviors. Content validation of the POMAQ was documented through qualitative interviews^17,18; however, the construct validity and test–retest reliability of the POMAQ has not yet been established.

The aim of this study was to validate the POMAQ following the recommendations of the FDA¹⁹. Specifically, this study was designed to evaluate the construct validity and test-retest reliability of the POMAQ to identify self-reported prescription opioid misuse and abuse behaviors among patients with chronic pain who were being treated with opioids. Construct validity is the degree to which a set of variables truly represents the construct that is to be measured²⁰. A key component of construct validity in this study was to evaluate whether patient self-report of behaviors could be supported by other validated questionnaires and objective measures. Test–retest reliability is the extent to which a measure produces stable scores over time in the same patients under similar conditions. Instruments must have adequate reliability to ensure that they are consistently measuring the construct of interest²¹.

Methods

Working definitions

The following definitions of “misuse” and “abuse” were used and are modified versions of definitions obtained from the Analgesic, Anesthetic, and Addiction Clinical Trials, Translation, Innovations, Opportunities, and Networks (ACTTION) review²².

Misuse: The intentional use of a drug for therapeutic purpose (to reduce an aversive symptom or state), inappropriately outside label directions or in a way other than prescribed or directed by a healthcare practitioner. This definition includes patients using a drug for a condition different from that for which the drug is prescribed, patients taking more drugs than prescribed, or at different dosing intervals.

Abuse: The intentional use of a drug for non-therapeutic purpose, repeatedly or sporadically, for the purpose of achieving a positive psychological or physical effect.

Study design

In this validation study, patients were recruited from five Department of Defense (DoD) Military Health System (MHS) clinics located across the US, including: Naval Medical Center Portsmouth (Portsmouth, VA); Naval Medical Center San Diego (San Diego, CA); Wright-Patterson Medical Center (Dayton, OH); Walter Reed National Military Medical Center (Bethesda, MD); and San Antonio Military Medical Center (San Antonio, TX). The MHS serves approximately 10 million beneficiaries who are generally demographically representative of the broader US population²³. Patients from clinics within each of the selected centers were identified and approached for enrollment. Potentially eligible patients were identified by site staff through an electronic medical records (EMR) database and were screened for study eligibility over the phone or in-person. Eligible patients had to be ≥18 years old, diagnosed with a chronic (≥3 months) pain condition and were being treated with opioids, willing to provide written consent and a urine sample, and able to participate in an internet-based survey and a telephone interview in English. Exclusion criteria included a diagnosis of a terminal illness as well as being a current, active-duty service member of the military.

Patients who were eligible and willing to participate provided written informed consent during an initial clinic visit. Each patient was assigned a unique participant study identification (ID) to be used to access the web-based portal for completion of the questionnaires. All recruitment procedures complied with the current Health Insurance Portability and Accountability Act (HIPAA) regulations. Each participating site was approved by an Institutional Review Board (IRB), and a Certificate of Confidentiality (CoC) from the National Institutes of Health was obtained.

Study measures

The construct validity of the POMAQ was examined by comparing data from it with data from: (1) a subjective patient-reported measure (the Prescription Drug Use Questionnaire – Patient Version [PDUQp]); (2) a mental health interviewer rating of substance abuse disorder (the Structured Clinical Interview for DSM-IV-TR Axis I Disorders [SCID-I]); (3) a subjective clinician-reported measure (the Physician Opioid Therapy Questionnaire – Version 2 [POTQ v2]); (4) claims and EMR data; as well as, (5) urine and hair screen data (see Supplemental Digital Content Table S1).

Patient-completed measures

Prescription Opioid Misuse and Abuse Questionnaire

The 19-item POMAQ was developed to identify the patient’s current and past behaviors related to prescription opioid misuse and abuse and the intent behind those behaviors as previously described¹⁸. For each affirmative response to a question, patients were asked additional questions regarding frequency and intention of the behavior. For this analysis, individual questions were used for the construct validity and reliability assessments, thus no scoring was needed. Scoring guidelines for the POMAQ are detailed elsewhere. Test–retest reliability was assessed by comparing the results of the POMAQ at two timepoints (∼7–10 days apart) among a subset of patients who were randomized to the retest administration of the POMAQ.

Prescription Drug Use Questionnaire – Patient Version (PDUQp)

The original PDUQ is an interviewer-administered questionnaire that includes 42 questions designed to assess abuse and misuse behaviors in patients with pain²⁴. The Patient Version of the PDUQ (PDUQp) was derived from the original tool and is composed of 31 questions that are self-completed by the patient²⁵. The PDUQp demonstrates moderate concurrent and predictive validity using the original PDUQ as criterion and has good test–retest reliability. Content areas of the PDUQp include pain condition, opioid use, family history of pain, patient history of substance abuse, and psychiatric history. PDUQp scores range from 0 to 30 and were calculated using the instrument’s scoring manual²⁵.

Although no questions are identical, several items of the PDUQp assess similar concepts to those found on the POMAQ (e.g. PDUQp question 17 [Have you ever lost your pain medications and needed them replaced?] and POMAQ question 12 [Has your prescription for opioid pain medication or your prescription opioid pain medication been lost?]). Other items are less directly similar, (e.g. PDUQp question 11 [Have you had to increase the amount of pain medications you take over the past 6 months?] and POMAQ question 4 [Did you take more of your prescription opioid pain medication than was prescribed to you?]). It is important to note that the recall period differs between the measures. The PDUQp asks about behaviors “ever” or in the past 6 months while the POMAQ asks about the past year and in the past 3 months.

Sociodemographic form

Participants also completed a brief questionnaire to document general sociodemographic information such as age, ethnicity, living situation, employment, and education. Additional patient-reported outcome assessments were also administered. These measures and their results are reported elsewhere [18].

Patient interview (administered by mental health expert)

Structured Clinical Interview for DSM-IV-TR Axis I Disorders (SCID-I) Substance Use Disorders Module.

The Structured Clinical Interview for DSM-IV-TR is a widely accepted diagnostic instrument to reliably determine Axis I disorders in non-patient and patient populations²⁶. Several studies have generated valid and reliable data related to substance use and abuse with this instrument^27–31. Patients were assessed for substance dependence and abuse using the Non-Alcohol Substance Use Disorder questions from the SCID-I Substance Abuse/Dependence interview module via a telephone interview by trained mental health experts within 10 days of completing the initial POMAQ survey. The SCID scoring algorithm was used to determine a diagnosis of abuse and dependence for substance use disorders. Concordance between responses to POMAQ items and participants’ SCID-I results to specific behaviors assessed in the SCID-I were assessed to examine the construct validity of the POMAQ. For example, participant report of marijuana use on the POMAQ was compared with the SCID-I assessment of marijuana use. Of note, the SCID-I does not distinguish misuse and abuse behaviors.

Clinician-completed measures

Physician Opioid Therapy Questionnaire – version 2 (POTQ V2)

The POTQ V2 is an 8-item scale completed by the treating physician to assess perception of misuse of opioids³². The treating clinician answered questions reflecting patients’ potential misuse and abuse behaviors, such as multiple unsanctioned dose escalations; lost or stolen prescriptions; frequent unscheduled visits to the pain center or emergency room; excessive phone calls; and inflexibility around treatment options. The POTQ has been found to be significantly correlated with the PDUQ and abnormal urine screens as an external measure of its validity³³.

Objective data

Urine and hair drug screening

All patients were asked to provide an unsupervised urine sample (as well as date and time of most recent opioid dose) during their enrollment visit. Patients who consented to providing the optional hair sample (scalp or body) provided a minimum of 100 mg during the enrollment visit as well. Concordance between participant responses to POMAQ prescription and illicit opioid use questions and results of participants’ urine and hair drug screens were assessed.

All urine and hair samples were analyzed using liquid chromatography-mass spectrometry/mass spectrometry (LC-MS/MS)³⁴ performed by Alere Toxicology (urine) (aleretoxicology.com) and Omega Laboratories (hair) (omegalabs.net). Many metabolites associated with various drugs, including but not limited to heroin, marijuana, methadone, fentanyl, oxycodone, codeine, benzodiazepine, hydromorphone, ketamine, MDMA, morphine, and numerous metabolites were tested for in the provided samples. Most substances of abuse can be detected in the urine for approximately 2–4 days from the time of administration, while hair samples can have a longer detection timeframe of four to six months³⁵.

EMR data

Medical records for one year prior to the consent date were obtained, which included patient-related information (i.e. dates of clinical visits, chief complaints, diagnoses, labs ordered, and prescriptions filled). POMAQ reports of healthcare resource utilization were compared with relevant EMR data, which included healthcare provider (HCP) visits, prescription opioid refills, and emergency room or urgent care visits. Given the range of healthcare provider types, visits were examined as two separate categories: (a) pain management, family physician, primary physician, and substance abuse clinic; and (b) all other HCP types inclusive of internal medicine, mental health, neurology, oncology, orthopaedic, paediatric, psychiatry, psychology, rehabilitation, specialty, unspecified, and others.

Procedures

Following written consent, patients were provided with a study packet including a welcome letter, study information, and directions on how to complete a series of questionnaires via a secure website. Once on the secure web-based portal, patients were provided further instructions on how to complete the POMAQ and other questionnaires at a time and location of their choice. During the initial in-clinic visit, all patients were asked to provide a urine sample. A separate, optional consent was sought from the patients to provide 100 milligrams (mg) of head or body hair. Clinic staff completed a case report form (CRF) for each patient, while the POTQ V2 was completed by patients’ enrolling physician.

Following the initial clinic visit, patients were given up to 21 days to complete the survey. After patients completed all the internet-based questionnaires, they received a $25 gift code via email. One to five days after survey completion, patients were contacted and asked to participate in a 30- minute telephone interview with a mental health expert interviewer. The interview was designed to screen for possible substance abuse disorders utilizing the SCID-I Substance Use Disorders Module. A second $25 gift code was provided to patients once they finished the interview.

Using a block randomization of 1:4, the POMAQ was administered a second time to 25% of the patients who had completed the first online survey. An email invitation with a link to the second online administration of the POMAQ was sent to eligible patients within 7–10 days following completion of the first survey. The second survey was completed in a secure environment of the patient’s choosing and another $25 gift code was sent upon completion. Test–retest reliability of the POMAQ was assessed by evaluating the concordance of the responses to the POMAQ items among the subset of patients who completed the second administration of the POMAQ and reported stable chronic pain. Stable chronic pain patients were defined as those who responded, “about the same” to the question, “Since your visit to the clinic, has there been any change in your chronic pain symptoms?” at the second administration of the POMAQ.

Patients’ survey responses were linked to their de-identified claims and EMR data using their unique ID and randomly generated registration code that was linked to their EMR number.

Statistical analyses

All statistical analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC). Descriptive statistics were used to describe the demographic and clinical variables.

As noted above, the construct validation process utilized objective data (e.g. hair and urine), observational data (e.g. EMR and SCID-I interviews) and patient self-report data (e.g. PDUQp) to assess the ability of the POMAQ to capture specific self-reported behaviors. Each behavior (i.e. POMAQ question) was validated following a priori developed algorithms against the relevant ancillary measures. Construct validity was evaluated using per cent agreement/concordance between specific POMAQ items and conceptually relevant items within these measures, except for the EMR claims data, for which the analyses were exploratory. Analyses included the prevalence-adjusted, bias-adjusted kappa (PABAK) statistic³⁶ which is thought to better handle the skewed responses noted. The PABAK statistic calculates its estimate of chance agreement using different assumptions to better adjust for high prevalence or bias in marginal distributions than the kappa statistic. PABAK values <0.20 are considered poor, while 0.2–0.60 is considered fair to moderate, 0.61–0.80 is good, and >0.80 is very good.

For the test–retest reliability analyses, the per cent agreement/concordance was calculated for each item between the first and the second administration of the POMAQ for a subset of patients who indicated they had stable chronic pain between survey administrations.

Results

Study sample

Of 3263 patients who were screened, 938 (28.7%) met eligibility criteria, consented, and were enrolled in the study. Of those screened, almost half of patients (n = 1589; 48.7%) chose to not participate in the study, 604 (18.5%) patients were considered ineligible due to a variety of reasons, and 133 (4.1%) were eligible and willing to participate but were not enrolled (due to scheduling difficulties and/or site logistics). A total of 809 (86.2%) patients completed the POMAQ. Of the remaining 129 patients who did not complete the POMAQ, 116 (89.9%) were not responsive to follow-up and 13 (10.1%) withdrew from the study.

Patients ranged in age from 20 to 88 years (mean age, 55.4 years), over half were female (55.5%), Caucasian (74.8%), and non-Hispanic (90.6%) (Table 1). The majority of patients were retired (28.6%), on disability (24.7%), or employed full-time (20.6%) and had some college (36.5%) or a college degree (33.7%). Comorbid conditions and current medication use have been described elsewhere³⁷.

Table 1. Participant-reported demographic and clinical characteristics.

Characteristic	Total	Test–retest stable sample
	(N = 809)	(N = 139)
Age, mean years ± SD	55.4 ± 12.7	53.7 ± 12.5
Range (min, max)	(20.0–88.0)	(20.0–81.0)
Gender, n (% female)	449 (55.5%)	81 (58.3%)
Ethnicity, n (%)
Not Hispanic or Latino	733 (90.6%)	6 (4.3%)
Hispanic or Latino	70 (8.7%)	133 (95.7%)
Missing	6 (0.7%)	0 (0.0%)
Race, n (%)
White or Caucasian	605 (74.8%)	111 (79.9%)
Other	93 (11.5%)	15 (10.8%)
Black or African American	84 (10.4%)	9 (6.5%)
American Indian or Alaska Native	10 (1.2%)	1 (0.7%)
Asian	9 (1.1%)	2 (1.4%)
Missing	6 (0.7%)	0 (0.0%)
Native Hawaiian or other Pacific Islander	2 (0.2%)	1 (0.7%)
Current pain condition (n, %)^a
Low back pain	620 (76.6%)	115 (82.7%)
Neck or shoulder pain	488 (60.3%)	84 (60.4%)
Osteoarthritis	313 (38.7%)	58 (41.7%)
Other neuropathic pain (nerve damage)	296 (36.6%)	52 (37.4%)
Migraine or other chronic headache	256 (31.6%)	39 (28.1%)
Other	224 (27.7%)	46 (33.1%)
Fibromyalgia	172 (21.3%)	31 (22.3%)
Post-operative pain	138 (17.1%)	21 (15.1%)
Rheumatoid arthritis	120 (14.8%)	18 (12.9%)
Central pain	111 (13.7%)	22 (15.8%)
Bone break or fracture	87 (10.8%)	22 (15.8%)
Painful diabetic neuropathy	78 (9.6%)	10 (7.2%)
Pain related to cancer	35 (4.3%)	6 (4.3%)
Phantom pain	29 (3.6%)	3 (2.2%)
Visceral pain	14 (1.7%)	3 (2.2%)
Post herpetic neuralgia	10 (1.2%)	2 (1.4%)
Missing	2 (0.2%)	0 (0.0%)
Sickle cell pain	1 (0.1%)	0 (0.0%)

Abbreviation. SD, standard deviation.

^aNot mutually exclusive.

Construct validity

PDUQp

The construct validity between POMAQ and PDUQp is shown in Table 2 as demonstrated by the concordance between the six similar items of the two measures. Questions that were worded most similarly or assessing the same concept between the POMAQ and the PDUQp showed the highest percentage agreement, ranging from 80.8% (POMAQ question 14 and PDUQp question 12) to 97.5% (POMAQ question 8 and PDUQp question 19). The PABAK values for these comparisons ranged from 0.616 to 0.949. The four related questions that were less similar in wording showed slightly weaker agreement, although concordance was still relatively high in those cases ranging from 73.7% (POMAQ question 4 and PDUQp question 11) to 88.3% (POMAQ question 15 and PDUQp question 18). Overall, the PABAK values for these comparisons ranged from 0.474 to 0.766.

Table 2. Construct validity of select questions on the POMAQ and PDUQp^a.

PDUQp item	POMAQ item		Percent agreement	PABAK
Q11: Have you had to increase the amount of pain medications you take over the past 6 months?	Q4. In past year…taken more prescription opioid pain medication than was prescribed?
	Yes (n = 156)	No (n = 627)
Yes (n = 202)	76 (9.7%)	126 (16.1%)	73.7%	0.474
No (n = 581)	80 (10.2%)	501 (64.0%)
Q12: Have you had to call in for more pain medications because your prescription ran out?	Q14. In past year…requested refills for prescription opioid pain medication earlier than they were due?
	Yes (n = 117)	No (n = 664)
Yes (n = 145)	56 (7.2%)	89 (11.4%)	80.8%	0.616
No (n = 636)	61 (7.8%)	575 (73.6%)
Q15. Do you ever use alcohol to help relieve some of the pain?	Q6. In past year…drink alcohol while taking prescription opioid pain medication?
	Yes (n = 229)	No (n = 556)
Yes (n = 81)	65 (8.3%)	16 (2.0%)	77.1%	0.541
No (n = 704)	164 (20.9%)	540 (68.8%)
Q17: Have you ever lost your pain medications and needed them replaced?	Q12. In past year… prescription opioid pain medication has been lost?
	Yes (n = 21)	No (n = 766)
Yes (n = 39)	17 (2.2%)	22 (2.8%)	96.7%	0.934
No (n = 748)	4 (0.5%)	744 (94.5%)
Q18: Have you had to visit the emergency room in the past 6 months because of your pain problem?	Q15. In past year… visited ER or Urgent Care clinic to get more prescription opioid pain medication?
	Yes (n = 29)	No (n = 758)
Yes (n = 109)	23 (2.9%)	86 (10.9%)	88.3%	0.766
No (n = 678)	6 (0.8%)	672 (85.4%)
Q19. Have you ever had to buy pain medications on the street?	Q8. In past year…got opioid pain medication from someone who was NOT a doctor or healthcare provider?
	Yes (n = 19)	No (n = 768)
Yes (n = 9)	4 (0.5%)	5 (0.6%)	97.5%	0.949
No (n = 778)	15 (1.9%)	763 (97.0%)

Abbreviations. PABAK, Prevalence-Adjusted Bias-Adjusted Kappa; PDUQp, Prescription Drug Use Questionnaire-patient version; POMAQ, Prescription Opioid Misuse and Abuse Questionnaire.

^aSample size based on patients who completed both PDUQp and POMAQ questions.

In general, PDUQp items had higher concordance with the 1-year recall questions of the POMAQ compared to the POMAQ 3-month recall questions (data not shown). This in part is attributed to the smaller number of patients who endorsed items with the shorter, 3-month recall.

POTQ V2

The POTQ V2 was completed by physicians for only 338 (36.0%) of the enrolled patients. Given the low rate of completion for this measure, its results are not reported here. Some of the reasons that physicians provided for not completing the POTQ V2 included: physician not comfortable completing the form (27.7%); and, physician out of network (5.8%). Approximately 35% of physicians cited “other” as the reason for not completed the POTQ but did not elaborate further.

SCID-I

The SCID-I interviews identified few substance abuse/dependence patterns among this patient sample (Table 3). Marijuana and alcohol use were the most frequently identified behaviors and the agreement of these behaviors with POMAQ responses was 98.5% and 99.8%, respectively. Five patients were found to have current prescription opioid abuse or dependence based on the SCID-I and all five of these patients also reported “yes” on the POMAQ to one or more of the following behaviors: having taken more opioids than prescribed, tampering with their medications, or taking opioids not prescribed to them in the past year. No patients were identified by the SCID-I for cocaine, hallucinogen, or illegal opioid abuse/dependence but a few patients (cocaine, n = 3; illegal opioids, n = 3; hallucinogen, n = 4) reported use of these substances on the POMAQ (data not shown). This suggests the POMAQ may be better at identifying low-level illicit behaviors than the SCID-I.

Table 3. Sensitivity and Specificity of Select Abuse/Dependence Items of the SCID-I Compared to the POMAQ^a.

SCID-I item	POMAQ item		Percent agreement	Sensitivity	Specificity
Current Alcohol Abuse or Dependence Evaluation	Q6. In past year… drank alcohol while taking prescription opioid pain medication?
	Yes (n = 6)	No (n = 436)
Yes (n = 7)	6 (1.4%)	1 (0.2%)	99.8%	85.7%	100.0%
No (n = 435)	0 (0.0%)	435 (98.4%)
Current Cannabis Abuse or Dependence Evaluation	Q9a. In past year, took prescription opioid pain medication with marijuana, hashish, or synthetic cannabis?
	Yes (n = 7)	No (n = 660)
Yes (n = 17)	7 (1.0%)	10 (1.5%)	98.5%	41.2%	100.0%
No (n = 650)	0 (0.0%)	650 (97.5%)
Current Prescription Opioid Abuse or Dependence Evaluation	In past year… Q4: took more than prescribed? Q5: tampered with medication? or Q8: got any opioid medication not prescribed to you?
	Yes (n = 5)	No (n = 587)
Yes (n = 5)	5 (0.8%)	0 (0.0%)	100.0%	100.0%	100.0%
No (n = 587)	0 (0.0%)	587 (99.2%)

Abbreviations. SCID-I, Structured Clinical Interview for DSM-IV-TR Axis I Disorders; POMAQ, Prescription opioid misuse and abuse questionnaire.

^aSample size based on patients who had current abuse or had no abuse based on the SCID-I evaluation.

Urine and hair drug screen

The mean elapsed time from the last opioid dose and time of urine collection was 22.4 (±113.8) hours with 25% of collection times being ≥12 hours after the last opioid dose. As such, the urine screen analysis results were truncated to only patients who had taken an opioid within 12 hours of providing a urine sample. Table 4 summarizes the concordance between POMAQ responses and the urine drug screen results for elapsed time since opioid taken ≤12 hours. Overall, the percentage agreement between POMAQ questions asking about medications used and urine screen results was higher when urine metabolites were analyzed by class of medication (e.g. benzodiazepines and opioids) versus specific individual medication report and its metabolites, indicating that patients may not be accurate in reporting their exact prescription medications. The percentage agreement for opioids as a class was 92.6% (PABAK, 0.853). Rates of POMAQ false negatives (Urine/hair screen: positive; POMAQ: negative) were generally low and less than 1% for most metabolites. In contrast, the POMAQ false positive rate (Metabolite: No; POMAQ: Yes) was 9.9% for benzodiazepines and 17.2% for opioids.

Table 4. Summary of construct validity between POMAQ and urine and hair toxicity screens.

Metabolites	POMAQ item		Percent agreement	PABAK
Urine metabolites for anti-anxiety medications^a	Responses to Item Q7b^b
	Yes (n = 159)	No (n = 631)
Yes (n = 119)	81 (10.3%)	38 (4.8%)	85.3%	0.706
No (n = 671)	78 (9.9%)	593 (75.1%)
Urine metabolites for opioid medications take n < 12 h from urine collection^c	Responses to Item Q1^d
	Yes (n = 596)	No (n = 2)
Yes (n = 556)	554 (92.6%)	2 (0.3%)	92.6%	0.853
No (n = 42)	42 (7.0%)	0 (0.0%)
Hair metabolites for anti-anxiety medications^e	Responses to Item Q7b^b
	Yes (n = 83)	No (n = 302)
Yes (n = 32)	24 (6.2%)	8 (2.1%)	82.6%	0.652
No (n = 353)	59 (15.3%)	294 (76.4%)
Hair metabolites for opioid medications^f	Responses to Item Q1^d
	Yes (n = 384)	No (n = 1)
Yes (n = 319)	318 (82.6%)	1 (0.3%)	82.6%	0.652
No (n = 66)	66 (17.1%)	0 (0.0%)

Abbreviations. 6-MAM, 6-monoacetylmorphine; CI, confidence interval; PABAK, Prevalence-Adjusted, Bias-Adjusted Kappa; POMAQ, Prescription Opioid Misuse and Abuse Questionnaire.

^a7-aminoclonazepam, alpha-Hydroxyalprazolam, Lorazepam, Nordiazepam, Oxazepam, Temazepam, Alprazolam, 2-Hydroxy-ethyl-flurazepam, 7-Aminoflunitrazepam, Alpha-Hydroxy-Midazolam, Estazolam, Diazepam.

^bQ7b: In the past 3 months, did you take your prescription opioid pain medication with [response from Q7]? (Response: Anti-anxiety medications (e.g. benzodiazepines such as diazepam [Valium®], alprazolam [Xanax^®], clonazepam [Klonopin^®], lorazepam [Ativan^®]).

^cCodeine, Morphine, Hydrocodone, Hydromorphone, 6-MAM, Norhydrocodone, Oxycodone, Oxymorphone, Noroxycodone, Naloxone, Dextromethorphan, Tramadol, o-desmethyltramadol, Buprenorphine, Norbuprenorphine, Fentanyl, Norfentanyl, Methadone, EDDP.

^dQ1: In the past 3 months, what prescription opioid pain medications have you taken that were prescribed to you by your doctor or healthcare provider? Responses: Hydrocodone alone or in combination with another medication; Hydromorphone; Oxycodone alone or in combination with another medication; Methadone; Codeine alone or in combination with another medication; Oxymorphone; Morphine; Fentanyl (e.g. Duragesic, Fentora, Abstral, Actiq, Lazanda, Onsolis, Subsys); Buprenorphine (e.g. Butrans, Subutex, Suboxone, Zubsolv, Belbuca, Bunavail); Tramadol alone or in combination with another medication; Tapentadol (e.g. Nucynta, Nucynta ER); Other opioids.

^eAlprazolam, alpha-Hydroxyalprazolam, Clonazepam, Diazepam, Nordiazepam, Oxazepam, Temazepam, Midazolam, 7-Aminoclonazepam, Nitrazepam, Triazolam, Lorazepam, Flunitrazepam, 7-Aminoflunitrazepam.

^fCodeine, Morphine, 6-MAM, Hydrocodone, Norhydrocodone, Hydromorphone, Oxycodone, Noroxycodone, Oxymorphone, Ketamine, Norketamine, Fentanyl, Norfentanyl, Sufentanil, Norsufentanyl, Meperidine, Normeperidine, Methadone, EDDP, Tramadol, Buprenorphine, Norbuprenorphine.

Approximately half of the patients provided hair samples (n = 429; 53%). Table 4 summarizes the construct validity between POMAQ questions asking about medications used and the hair screen results. Similar to the urine screen results, the percentage agreement between POMAQ questions and hair screen results was high when hair metabolites were analyzed by class of medication (i.e. benzodiazepines and opioids). The percentage agreement was 82.3% (PABAK, 0.647) for benzodiazepines. False-negative and positive results provided in Table 4 were similar to those observed in the urine toxicity analyses and generally low.

Electronic medical records

Healthcare provider shopping

For the POMAQ question regarding visiting more than one doctor or healthcare provider (HCP) in the past year to get more prescription opioid medication, 89 of 809 patients (11%) responded “Yes” (Table 5). Based on EMR data counts of recorded visits for a 1-year recall, mean (± SD) number of visits to pain management HCPs, family physicians, primary physicians, and substance abuse clinics were 6.1 (± 9.1) in the “Yes” group and 5.2 (± 8.6) in the “No” group. When visits to all HCP types, which may or may not have involved getting opioid medications, were evaluated, the means (SD) increased to 28.1 (± 25.2) for the “Yes” group and 24.9 (± 24.9) for the “No” group indicating that clinical visits of all types did not differentiate answers to the POMAQ question about visits to get more opioid medications.

Table 5. EMR data by POMAQ response.

EMR data point	POMAQ item
“HCP shopping” EMR data	Q10: In the past year…visited more than 1 doctor or healthcare provider to get more prescription opioid pain medication?
	Yes	No
	(N = 89)	(N = 720)
Mean # of visits to pain management, family physician, primary physician, or substance abuse clinics (SD)	6.1 (9.1)	5.2 (8.6)
Mean # of all HCP^a visits (SD)	28.1 (25.2)	24.9 (24.9)
“Pharmacy shopping” EMR data	Q11: In the past year…gone to more than 1 pharmacy to obtain prescription opioid pain medication?
	Yes (N = 215)	No (N = 594)
# of Pharmacy locations for new prescription opioid (mean, SD)	1.40 (0.50)	1.21 (0.43)
# of pharmacy location for opioid refill (mean, SD)	1.47 (0.53)	1.17 (0.39)
Patients with at least one refill from a different pharmacy (N [%])	98 (46%)	95 (16%)
Early prescription refill EMR data	Q14: In the past year…requested refills for prescription opioid pain medication earlier than they were due?
	Yes (N = 122)	No (N = 687)
# opioid refills in past 1 year (mean, SD)	14.9, 7.8	12.2, 7.2
Patients with at least one refill < 21 days from prior prescription (days’ supply > 21 days) (N [%])	56 (45.9%)	200 (29.2%)
Patients with 15 or more refills for an opioid (N [%])	10 (8.2%)	18 (2.6%)
ER/UR EMR data	Q15. In past year… visited ER or Urgent Care clinic to get more prescription opioid pain medication?
	Yes (N = 32)	No (N = 777)
# ER/UC clinic visits in past 1 year (mean, SD)	2.3 (2.3)	1.1 (2.7)

EMR, electronic medical record; ER, emergency room; HCP, healthcare provider; POMAQ, Prescription Opioid Misuse and Abuse Questionnaire; SD, standard deviation; UC, urgent care.

^aPain management, family physician, primary physician, substance abuse clinics. Internal medicine, mental health, neurology, oncology, orthopedic, paediatric, psychiatry, psychology, rehabilitation, specialty, unspecified, and others.

Pharmacy shopping

A total of 215 patients responded that they had gone to more than one pharmacy to obtain a prescription opioid pain medication in the past year while 594 patients responded “No” (Table 5). According to EMR data, more patients in the “Yes” group (n = 98; 46%) obtained refills from more than one pharmacy than those in the “No” group (n = 95; 16%). Additionally, the mean number of pharmacy locations for new prescription opioid medications and the mean number of pharmacy locations visited for opioid refills were similar between the “Yes” and “No” groups.

Early prescription refills

A total of 122 patients reported on the POMAQ that they had requested refills for their prescription opioid medications earlier than the prescription medications were due while 687 patients said they had not (Table 5). Among the group who responded “Yes” to this question, 56 (45.9%) patients had at least one instance of refilling their opioid prescription < 21 days from prior prescription in their EMR versus 29.2% of patient who responded “no” to this question. Ten patients (8.2%) who said “Yes” to early refill requests had 15 or more refills for an opioid versus 2.6% who responded “No.”

Emergency room and urgent care visits

Thirty-two patients (4.0%) reported visiting the emergency room (ER) or urgent care in the past year (Table 5). Among these patients, the mean (± SD) number of visits was 2.3 (± 2.3) as documented in their EMR data. Among those who reported not visiting the ER or urgent care for pain in the past year (n = 777), there was a lower number of ER and urgent care visits (mean ± SD = 1.1 ± 2.7).

Prescription fills

Patient responses to POMAQ question 1 (In the past 3 months, what prescription opioid pain medications have you taken that were prescribed to you by your doctor or healthcare provider?) were compared to the participant’s EMR prescription opioids fills within the last three months. The percentage agreement between what the patients reported as their current opioid medication and what was in their EMR was extremely high and ranged from 87.9% (oxycodone) to 99.8% (oxymorphone) with most having agreement greater than 94% (data not shown). For opioids that are prescribed more infrequently (e.g. oxymorphone, methadone, and buprenorphine), the percentage agreement was 99% or greater.

Reproducibility

Overall, the 19 POMAQ items demonstrated excellent test-retest reliability. Mean time between the first and second POMAQ administrations was approximately seven to 10 days apart (mean ± SD; 8.7 ± 2.3). The percentage agreement/concordance and PABAK statistic were calculated on each item between the first and second administration of the POMAQ among stable patients (defined as those who responded, “about the same” to the question, “Since your visit to the clinic, has there been any change in your chronic pain symptoms?” at the second administration of the POMAQ). The stable sample for the reproducibility assessment consisted of 139 patients (84% of retest sample).

Among patients with stable chronic pain, the test–retest reliability for all POMAQ questions was high (>90%) for all but three questions (i.e. taking less opioids [item 3], taking more opioids [item 4], and visiting more than one pharmacy [item 11]). PABAK values were very good (>0.80) for all but the same three questions (though these PABAK values were still good and ranged from 0.640 [item 3] to 0.755 [item 11]). For non-multi-response questions, the percentage agreement ranged between 82% (i.e. taking less opioids) and 100% (questions regarding stolen opioids, using a fake or stolen prescription, or stealing a prescription pad). For multi-response items, test-retest reliability was again excellent, with per cent agreement ranging between 84.9% (antihistamines) and 99.3% (barbiturates) for item 7 (i.e. medications prescribed in the past year). Similarly, for POMAQ item 9 (i.e. “street” drugs used in the past year), per cent agreement ranged between 87.8% (none) and 100% (hashish, synthetic cannabis, anabolic steroids, amphetamine, methamphetamine, cocaine, MDMA, GHB, flunitrazepam, dextromethorphan, phencyclidine, ketamine, inhalants, LSD, mescaline, 5-MeO-DMT, psilocybin, substituted phenethylamine, prescription-strength cough syrup, heroin, opium, and desomorphine).

Discussion

This study was designed to examine the construct validity and reproducibility of the POMAQ, a self-report measure of prescription opioid misuse and abuse behaviors using patient- and clinician-reported assessments, objective urine and hair drug screening results, and EMR records. The evidence presented suggests that the POMAQ demonstrates good construct validity and that patient’s self-reporting behaviors were corroborated by other subjective (PDUQp and SCID-I) and objective (urine/hair data, and EMR) reports. Furthermore, the POMAQ also demonstrated excellent test-retest reliability among stable chronic pain patients who completed two POMAQ administrations.

Specific POMAQ items were assessed against items on the patient-completed PDUQp that assessed similar concepts. While there was a substantial degree of agreement supporting the validity of the POMAQ, the percentage agreement was not uniformly high. This is a common finding in validation studies, and it is thought to be due to the different wording of the questions (i.e. no questions between the two instruments were identical) and the different recall periods between the two questionnaires. This is supported by the fact that the correlation was strongest for items that were similarly worded between the POMAQ and PDUQp, with weaker support for items in the two measures that assessed similar concepts but were worded and posed in a slightly different context. Additionally, the recall period for the POMAQ items is 1-year, initially, with a recall period of 3-months as a follow-up question, whereas the recall period of the PDUQp is 6-months. As such, the lack of consistent recall period between the PDUQp and POMAQ, which may have resulted in discrepancies in patient responses. Even so, the correlation between questions was acceptably high for most questions. These results highlight the differences in the two questionnaires as they are capturing different behaviors related to prescription opioid misuse and abuse.

The SCID-I interviews were strongly supportive of the POMAQ. Importantly, there were very few cases where the SCID-I identified substance abuse or dependence that were not reported on the POMAQ. The POMAQ appears to identify behaviors that have patterns of use that were not detected by the SCID-I level of diagnosis for substance abuse or dependence. With more behaviors reported on the POMAQ than identified by the SCID-I, the POMAQ is likely to be better at identifying behaviors than the SCID-I. The one drug that was captured more frequently by the SCID-I than the POMAQ was the use of cannabis. Ten patients were identified by the SCID-I as having cannabis dependence, but not on the POMAQ. This could be due partly to the wording of POMAQ, which positions marijuana as a street drug and does not reflect the possibility of marijuana being obtained legally. The POMAQ has since been revised to ask a separate question regarding marijuana use and whether it was obtained illegally or legally (either over-the-counter when available or by prescription).

Drug screening is usually conducted through urine testing because of its low cost, however there is a relatively short window of time to detect drug metabolites in urine, although it is longer than the window for detecting metabolites in the blood. Drug screening through hair testing can capture drug use over a period of 3 or more months³⁸; however, such testing also has limitations. In the current study, patients appeared to report their medication use accurately as corroborated by both hair and urine screens. For some medications, more patients reported using a drug that was not in their urine or hair results (i.e. false-positive rates were higher than false-negative rates). Additionally, given that the duration of time of urine sample collection was on average 22.4 hours from the patients’ last dose of opioid, the results of analyses for urine were truncated to include only patients who had taken an opioid within 12 hours of urine collection. Another likely reason for the higher false-positive rates was the disconnect between the 3-month recall period for medication use of the POMAQ and the point in time when urine and hair samples were collected. Within the 3-month period that patients were asked to recall, they may have been prescribed, or used, multiple medications, which may have been discontinued prior to the time of study enrollment and urine/hair sample collection.

As this patient population is one with many chronic conditions in addition to their chronic pain, with high resource utilization rates, opioid refills and pharmacy behaviors provided a more meaningful and relevant comparison with POMAQ responses than healthcare provider visits. Given the high prevalence of HCP visits, it is difficult to determine the relevance of HCP visit counts as related to opioid seeking versus other healthcare needs. It should be noted that the EMR data are limited in that they reflect only what was processed through the DoD insurance system. Thus, if a patient sought care or refilled prescriptions outside of the DoD system, these specific events would not be captured in the EMR data.

When EMR prescription data for opioids were compared with patient reports of what medications they had been taking over the past 3 months, the percentage of agreement was high. However, as with the urine/drug screens and POMAQ responses, mismatches occurred between what patients reported and what was actually prescribed over the past 3 months as documented in their EMR. This provides additional evidence that patients may not always know the correct name of the medications they are taking. It may also be a reflection that patients often have multiple prescriptions for opioid medications and that their prescribed treatments changed over the course of three months.

There are limitations of the study that need to be considered. It is important to recognize that patients with chronic pain frequently have other chronic conditions, which typically result in high resource utilization rates. Of the healthcare resource use examined, opioid refills and pharmacy behaviors were the most meaningful and relevant parameters for evaluating POMAQ responses since these are specific to patients’ chronic pain management. Healthcare visits were less relevant as patients often visited multiple healthcare providers for multiple indications which may not be related to asking for prescriptions from different providers, thus, making it difficult to determine the relevance of HCP visit counts as related to opioid seeking. Additionally, the EMR data examined are limited as they reflect only what was captured in the DoD database. The nature of the topic of misuse and abuse of prescription opioids is highly sensitive. Self-reporting on illicit behaviors is a difficult and uncomfortable task for most patients; providing an environment in which capturing honest responses on tools such as the POMAQ is challenging. Finally, it is possible that patients declined to participate in the study once they realized a urine sample was required. This inclusion criterion may have created a selection bias by deterring those who may have recently engaged in problematic drug use patterns, thereby enriching the sample with individuals who did not engage in such behaviors.

The provision of primary care is often not structured to systematically screen and treat chronic pain, let alone screen for harmful behaviors associated with prescription opioid use. High volume primary care centers are faced with particular challenges in terms of constraints on clinic workflow and limited staff and physician time and the lack of on-site resources for interdisciplinary care. Thus, tools that facilitate rapid assessment, enhance patient–physician communication, and target interdisciplinary goals are essential to advancing chronic pain management, optimizing patient care, and identifying potential harmful behavior(s) related to prescription drug use. Valid, standardized screening measures are critically needed, especially in disease areas as prevalent and impactful as chronic pain.

The POMAQ fills a critical measurement gap for the screening of prescription opioid misuse and abuse behaviors. The self-administration of the POMAQ makes this questionnaire easier to implement as a patient monitoring instrument than interviews such as the SCID-I. Additionally, the assessment of intentionality behind a behavior assists in determining whether that behavior is misuse or abuse. Better understanding of the purposes behind aberrant medication-related behaviors may help generate interventions directly targeted to reducing misuse and abuse behaviors. Furthermore, secure, online administration of the POMAQ may facilitate honest patient self-report of misuse and abuse behavior, which is critical for effectively identifying and managing prescription opioid misuse and abuse. Given the potential dangers and harmful behaviors associated with long-term prescription opioid use, monitoring behaviors at regular intervals systematically in the healthcare system may assist in avoiding further exacerbation of the national opioid crisis. The POMAQ may serve as a bridge between the critical role played by prescription opioids in pain reduction and the urgent need for screening and monitoring of behaviors associated with such medications. For the POMAQ to serve as the bridge described above and to improve patient outcomes, there must be sufficient uptake of the measure in clinical practice. Therefore, reducing barriers to its implementation in everyday practice and providing appropriate education on the tool are important steps to ensuring uptake of the POMAQ.

Conclusions

This study demonstrated a high level of construct validity and reliability of the POMAQ using both subjective patient- and clinician-reported measures as well as objective data in a population of patients with chronic pain who were receiving prescription opioids. The data from the PDUQp, SCID-I, urine and hair drug screens, and EMRs suggest that patients are honestly self-reporting and that the POMAQ is a valid and reliable tool that can be used to assess the presence of prescription opioid misuse and abuse behaviors among patients with chronic pain. Further evaluation needs to be conducted to assess the longitudinal usefulness of the POMAQ in detecting change in behaviors over time.

Transparency

Declaration of funding

This project was conducted as part of a FDA-required post-marketing study for extended-release and long-acting opioid analgesics and was funded by the Opioid Postmarketing Consortium (OPC) consisting of the following companies: Allergan; Assertio Therapeutics, Inc.; BioDelivery Sciences, Inc.; Collegium Pharmaceutical, Inc.; Daiichi Sankyo, Inc.; Endo Pharmaceuticals, Inc.; Hikma Pharmaceuticals USA Inc.; Janssen Pharmaceuticals, Inc.; Mallinckrodt Inc.; Pernix Therapeutics Holdings, Inc.; Pfizer, Inc.; Purdue Pharma, LP.

Declaration of financial/other relationships

Research data derived from approved Naval Medical Research Unit-Dayton, Dayton, Ohio and Naval Medical Center, Portsmouth, Virginia IRB, protocol; number NAMRUD.2015.0004. The views expressed in this article are those of the author(s) and do not necessarily reflect the official policy or position of the Department of the Navy, Department of Defense, or the United States Government. At the time of this work, CAPT AA Bukhari was a member of the U.S. Military. This work was prepared as part of his official duties. Title 17U.S.C. 105 provides that “Copyright protection under this title is not available for any work of the United States Government.” Title 17U.S.C. 101 defines a United States Government work as a work prepared by a military service member or employee of the United States Government as part of that person's official duties.

K. S. Coyne and B. M. Currie are employees of Evidera, a company that received funding from the OPC to conduct this study. At the time of the study, A. I. Barsdorf was employed by Pfizer, a member company of the OPC. At the time of the study, J. Y. Mazière was an employee of PPD, a parent company to Evidera and is now an employee of Astellas. R.F. Pierson is an employee of Janssen, a member company of the OPC. S. H. Schnoll is an employee of Pinney Associates, a company that received funding from the OPC for time spent consulting on this study. S. F. Butler received funding from the OPC for time spent consulting on this study. J. T. Farrar is an employee of the University of Pennsylvania and received funding from the OPC for time spent consulting on this study to support his salary. H. J. Fisher is an employee of Health ResearchTx, a company that received funding from the OPC to conduct this study. CAPT A.A. Bukhari, USAF has no financial disclosures to report. All aspects of the study design, interpretation, and decision to submit for publication were determined by the authors.

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgements

The authors thank Kawthar Nakayima, Amara Tiebout, and Michael Grossi for their production and editorial assistance. The authors would also like to thank Kim Poinsett-Holmes, PharmD (Evidera, Morrisville, NC), Mahesh Paschapur (Evidera, New Delhi, India), and Amara Tiebout (Evidera, Bethesda, MD) for medical writing/editorial assistance.