https://orcid.org/0000-0002-3746-9529
ABSTRACT
Objective: Vaccine hesitancy is a complex problem. We previously demonstrated that motivational interviewing (MI) could be helpful to enhance parents’ motivation to vaccinate their child. The aim of this study is to develop a new, simple and robust evaluation tool that is suitable for evaluating MI learning of vaccination health professionals.
Methods: We designed the Motivational Interviewing Skills in Immunization (MISI), a short written questionnaire to evaluate the MI knowledge and skills of participants in an immunization context. It covers three key areas: knowledge of MI, ability to apply MI-related skills, participant self-confidence in using MI. Questionnaire content and face validity were assessed by MI experts and internal consistency, reliability and effect size were analyzed using a multiple pretest-posttest design.
Results: Psychometric measures showed good to excellent internal consistency of the questionnaire for all three areas (Cronbach’s and KR coefficient: 0.70 to 0.88). Test-retest reliability showed good measurement stability (ICC: 0.53). Good sensitivity to change was also obtained (Cohen’s d: 0.80 to 1.66).
Conclusion: The MISI questionnaire is the first paper/pencil evaluation method to assess MI training specific to immunization. Psychometric measures showed high reliability.
Practice implications: This questionnaire could provide a convenient and inexpensive method to evaluate knowledge and competencies following immunization-specific MI training.
Introduction
Despite vaccine hesitancy being a global and complex problem decried by the World Health Organization, the vast majority of vaccination promotion strategies don’t modify attitudes and behaviors towards vaccination.Citation1-Citation8 To address this problem, we developed a new strategy to promote vaccination, using motivational interviewing (MI) with parents of newborn infants.Citation9-Citation12 MI is a communication technique commonly used for addiction treatment,Citation13,Citation14 and increasingly so in health promotion, including immunization.Citation11,Citation12,Citation15-Citation21 MI is a patient-centered and directive counseling style, designed to help reinforce a patient’s motivation to change a behavior.Citation22-Citation24 Our previous study showed that a short MI-based intervention significantly increases parents’ intention to vaccinate their child and significantly improves vaccine coverage of these children.Citation9-Citation12 With the parents of newborn infants, MI is a convenient and inexpensive way to address both immunization concerns and benefits, in order to help raise motivation to vaccinate offspring.
Given the promising results of our previous study, we developed immunization-specific MI training for nursing health professionals administering vaccines in health facilities, homes, schools, and the community, as nurses mainly perform all vaccinations in Quebec, with an aim to promote vaccination among their patients. The first of day of this MI training is devoted to discovering the MI spirit and acquiring the basic MI skills using theory, observation and active participation of trainees in situational exercises in the context of immunization. After a three-month interval, the second day of training is devoted to revisiting MI skills learned on day and providing feedback to participants. This immunization-specific MI training is based on recurring fears and myths about vaccination where all examples used in the clinical cases are related to immunization. In this specific training, learning and use of the MI skills are tailored according to the patients’ vaccine hesitancy level.
Although this communication technique appeals to most health practitioners and MI training is gaining in popularity,Citation25,Citation26 certain concerns regarding the evaluation methods of MI skills post training need to be addressed. Standardized validated instruments such as the Motivational Interviewing Treatment Integrity Code (MITI),Citation27-Citation29 the Behaviour Change Counseling Index (BECCI),Citation30 the Motivational Interviewing Skills Coding System (MISC)Citation31 and the Motivational Interviewing Supervision and Training Scale (MISTS)Citation32 already exist and are used to assess MI competencies of trainees in any health topic. Most of the current validated tools for MI skill evaluation involve recorded audio or video sessions of the patient-provider encounter. Not surprisingly, obtaining consent from patients can be challenging. Some difficulties related to the audio or video equipment can also be experienced by the health care practitioners such as access to the material from the institution, poor sound quality and IT security policies. In addition, most evaluation tools are not suitable for large-group evaluation of MI skills following training. Indeed, most are relatively complex and time consuming tools for scoring the items needed to evaluate MI skills among individuals. Some of these tools also require extensive training of the part of the evaluators, an extra challenge for large-group studies requiring several evaluators.Citation33
To our knowledge, few validated evaluation tools currently exist in the written form to assess the MI learning and skills of trainees. While the Motivational Interviewing Knowledge and Attitude Test (MIKAT) is a written survey assessing knowledge and attitudes about MI, it is limited by the fact that MI competencies are not evaluated.Citation34 Few authors suggest written surveys based on the Helpful Response Questionnaire (HRQ) to evaluate MI-related skills, instead using a different coding method adapted for MI-related skills.Citation35,Citation36 HRQ is a written open-response questionnaire to evaluate respondent empathy,Citation37 providing an efficient means for assessing respondent ability to generate a response relevant to an open-ended question instead of respondent capacity to discriminate between answers from a list. The HRQ has been validated for use with groups of professionals. As very few validated MI-skill evaluation tools that are suitable for use with large groups, are immunization-specific, the aim of this study is to develop such a written evaluation tool. Ideally, this new written tool should provide a convenient, robust and reliable questionnaire to evaluate the MI knowledge and skills of vaccination professionals, as well as their ability to generate adequate answers in a situational exercise reflecting MI integrity.
Results
Content and face validity
A panel composed of four MI experts identified 18 of our original 32 items as less relevant across all three areas. Of these, 5 were dropped from MI knowledge assessment, 1 from the open-response questions, 11 from ability to apply MI skills, and 1 from self-confidence in using MI skills in clinical practice. Four additional items needed clarification according to the expert panel: 1 in the area of MI knowledge and 3 in that of MI skills. Modifications were made as suggested by the experts. According to the expert panel’s recommendations, 21 new items were written into the questionnaire by our team: 5 for MI knowledge, 12 for MI skills and 4 for self-confidence in using MI skills in clinical practise.
Experts recommended reducing the two open-response questions to a single one to encourage participants to provide a detailed response, instead of two shorter responses. The expert panel also suggested including new clinical cases for the open-response question in each of the three follow-up questionnaires to ensure participants did not provide identical responses to posttest questionnaires.
At the end of this revision process, the new questionnaire contained a total of 31 items: 9 multiple-choice questions for MI-knowledge acquisition, 1 open-response question and seventeen 6-points Likert-scale items for MI-skills application, and 8 items assessing self-confidence in using MI skills in clinical practice.
Participation
Three cohorts of nursing health professionals enrolled for our vaccination-specific MI training in 2017 for a total of 37 nurses, three (3) of whom did not complete the entire training. Among these, 35/37 (94.6%) and 22/34 (64.7%) completed the questionnaire prior to and after the first day of training, respectively. Two months later (i.e. before the 3-month half-day training course), 32/34 (94.1%) completed the questionnaire. Finally, following the second half-day training workshop, 26/34 (76.5%) completed the questionnaire.
Internal consistency
Internal consistency was calculated prior to day 1 and after day 2 for two area (). Cronbach’s α for the open-response item was 0.70 for both measures, which represents good reliability. The Kuder-Richardson coefficient (KR20) for the categorical items of the MI-skills area was low for the measure at pre-day 1 (KR20 = 0.574). After deleting 2 items that correlated less to other items of this area, the 15 remaining items showed good consistency; KR20 = 0.71 and KR20 = 0.79 at pre-day1 and post-day 2, respectively. Opposite results were observed for the last area; Cronbach’s α were too high at pre- and post-training (α = 0.92 and α = 0.91, respectively). According to analyses, two redundant items were removed, resulting in an excellent internal consistency: α = 0.87 for pre-day 1 and α = 0.88 for post-day 2.
Table 1. Internal consistency for MI-skills application and self-confidence to use MI
Following this analysis, the final questionnaire contained 9 multiple-choice questions for MI-knowledge acquisition, 1 open-response question and fifteen 6-points Likert-scale items for MI-skills application, and 6 items assessing self-confidence in using MI in clinical practice (See Supplementary Data).
Test-retest reliability
Twenty-one (21) participants answered the items from the MI-knowledge section after training on day 1 and two months later. Mean scores were not significantly different after the 2-month interval (p = 0.565) (). Scores of this section showed good test-retest reliability, as the intra-class correlation coefficient (ICC) was 0.53.
Table 2. Mean scores of MI knowledge at post-day 1 and 2 months later
Intra- and inter-coder reliability
Excellent intra- and inter-coder reliability was calculated for the mean score of the open-response question. Two independent MI-expert evaluators coded 19 random questionnaires and the ICC was 0.74. Two weeks after coding all questionnaires, a single MI-expert coder was asked to recode 19 random questionnaires and the ICC was 0.76.
Effect size
Scores at pre-training day 1 and post-training day 2 were significantly different across each area of the questionnaire (). The MI-knowledge section showed the greatest effect size (Cohen’d = 1.66) while self-confidence in using MI in the clinic showed the weakest (Cohen’s d = 0.80), although all these results showed very good sensitivity to change.
Table 3. Mean scores on each questionnaire area at pre-day 1 and post-day 2
Discussion and conclusion
Discussion
In this study, we propose a new written questionnaire to evaluate MI-knowledge acquisition and MI-skills application, following immunization-specific MI basic training. Our questionnaire design allowed evaluating three key aspects: MI-knowledge acquisition, ability to apply MI-related skills and self-confidence in using MI skills in clinical practice. MI-related skills were assessed using two different types of evaluation: a) by writing a detailed MI-consistent real-life dialogue in answering an open-response question and b) by self-rating the frequency of MI-adherent and non-adherent behaviors. These key aspects are consistent with two MI training outcomes described in the Kirkpatrick’s training evaluation model, which are clinical use of MI and MI competencies as the trainee improved ability to use MI elements.Citation38 The open-response question written in the form of a dialogue is a relatively convenient and inexpensive method of evaluation to measure the use of MI in clinical practice. Although the MITI is currently the gold standard to assess MI skills in clinical practice, it is however more complex and quite expensive to administer, with 20-minute audio recordings to code for each trainees. The open-response item of our questionnaire allowed participants to write down what they would respond to a patient in a real-life situation using their MI competencies according to the MI-spirit. This paper/pencil method is convenient for use with large-groups. Therefore, our tool seems highly feasible for MI training workshops and multiple-group trainings (such as our MI training given three times a year) where the use of the gold standard evaluation method, such as MITI, is unlikely to be feasible. With our coding protocol inspired from the MITI scoring,Citation27 it is also easier and faster to code the text response of several participants. Other studies also adapted the HRQ tool and developed their own coding system.Citation35,Citation39 In this case, we prepared a coding manual for our situational vaccination-specific open-response question so that other coders can evaluate the text response to this question (See Supplementary Data).
According to Kirkpatrick’s training evaluation model, two other MI training outcomes should be assessed. Namely, participants’ reactions to the training and patient health outcomes. We assessed the former by satisfaction surveys administered at the end of each of the training days and very high level of satisfaction was expressed from participants for both days (94% and 93%, respectively). The latter cannot be assessed in this study since the questionnaire only targeted trainees and not patients. However, we have assessed patient health outcomes previously.Citation9,Citation10,Citation12 Briefly, we showed that parents exposed to an immunization-specific MI intervention had greater intention to vaccinate their offspring and that these children display improved short-term vaccine coverage.
In addition to be the first questionnaire evaluating immunization-specific MI training, this new questionnaire was validated by a panel of international MI experts. Review of the questionnaire by this panel of MI experts was highly useful in discriminating between relevant and non-relevant items, and in order to make appropriate modifications and additions. Their contribution was invaluable since they suggested assessing all MI components such as MI principles, MI processes, MI spirit and MI skills, using several types and sets of questions. For instance, the section covering MI knowledge contains one set of items written in a theoretical manner, and another written in the form of a clinical case. After validation by the MI panel of experts, several analyses were performed to demonstrate the questionnaire’s reliability and responsiveness.
Psychometrics
Internal consistency analyses resulted in the deletion of non relevant items from the section on the ability in applying MI-related skills, as well as that of redundant items from the section covering the self-confidence in using MI in the clinic. The remaining items showed good reliability, as Cronbach’s α and Kuder-Richardson coefficient ranged between 0.70 and 0.90.Citation40 Internal consistency was not analyzed for the MI-knowledge section since its aim was not to discriminate between trainees’ level of MI knowledge. To find out if a majority of participants had increased MI knowledge after the training, we asked trainees to answer 9 relatively easy items. We analyzed test-retest reliability between results measured after day 1 and 2 months later. For this section, the questionnaire showed acceptable to good measurement stability over time as ICC ranged between 0.4 and 0.75, as previously described by Fleiss.Citation41 Test-retest reliability was not assessed for the other two areas covered by the questionnaire, as we expected participants to improve their MI skills and self-confidence in using MI over time.
Intra- and inter-coder reliability calculated for the open-response question was excellent, as suggested by Fleiss.Citation41 This result is consistent with other studies who adapted the HRQ items for written evaluation of MI skills.Citation35,Citation39 The inter-coder reliability result confirms that our coding protocol for the open-response is clear and precise. Indeed, a second coder similarly scored the same text responses compared to the first coder.
For each area covered by the questionnaire, a large effect size was observed. For MI knowledge and the two types of questions for MI-related skills, the difference between mean scores from pre-day 1 and post-day 2 was one standard deviation higher. Although Cohen’s d did not reach one standard deviation, questionnaire responsiveness for the “Self-confidence in using MI” section was also qualified as excellent according to Cohen and McDowell.Citation42,Citation43
Limits
One limit of this study is the absence of a control group. Without a control group, it is difficult to determine if the findings related to MI-skill acquisition are exclusively related to the training protocol or if other factors contributed. However, our questionnaire was completed at 4 different time points during the training, in order to adequately represent trainee learning. Although our written questionnaire can evaluate vaccination-specific MI knowledge and MI-related skills, it cannot show whether there is an actual behavioral change associated to the MI training of the nursing health professionals. Comparison of our questionnaire to the MITI is currently under investigation and will be the object of a forthcoming study. We expect that MISI questionnaire is probably not as precise as the MITI. However, the MISI is more amenable to large-group training and multiple-group training because of its high level of convenience and its low level of cost to use.
Conclusion
The MISI questionnaire is the first paper/pencil evaluation method to assess immunization-specific MI training. Preliminary psychometric measures showed high reliability and future validation of our questionnaire against a gold standard tool will confirm its use as an efficient, convenient and inexpensive evaluation method.
Practice implications
With the increasing proportion of vaccine-hesitant individuals, immunization-specific MI training will offer to immunization providers an effective approach to communicate with vaccine-hesitant patients and promote vaccine uptake. We foresee this questionnaire could be used from now on as a rapid and inexpensive method to evaluate MI-knowledge acquisition and competencies following immunization-specific MI training. This tool can be used among all healthcare professionals administering vaccines and is available as Supplementary Data.
Methods
Training evaluation
We used a multiple pretest-posttest design to evaluate our training protocol. We asked trainees to complete a pretest and a posttest before and after each day of the two-day training. We designed a short written questionnaire to evaluate three key aspects of the participants’ MI training: 1) knowledge of MI theory and principles, and MI-related skills, 2) ability to apply MI-related skills, and 3) self-confidence in using MI in clinical practice.
Questionnaire design
In order to evaluate our immunization-specific MI training protocol, we designed a short written questionnaire to be administered before and after each day of training. Following a careful review of relevant scientific literature on MI evaluation, a preliminary questionnaire was drafted according to the three content areas identified in published studies:Citation35-Citation39,Citation44 1) MI knowledge acquisition, 2) MI skills application and 3) confidence in practising MI. These three main areas were used to design the Motivational Interviewing Skills in Immunization (MISI) questionnaire. The MISI questionnaire items were constructed de novo in collaboration with a team of experts from the Motivational Interviewing Network of Trainers (MINT) and immunization experts involved in the PromoVac strategy.Citation9,Citation12 Invitation to participate was sent to several experts and those who were interested were selected in the panel. The first area – MI knowledge, theory and principles, and MI-related skills – is assessed by 9 multiple-choice questions. The next area – ability to apply MI-related skills – was assessed using two separate types of questions. The first set was inspired from the HRQ in order to assess the application of MI-related skills.Citation37 This set consists of 2 vaccination-specific situational, open-response questions that participants are instructed to answer by writing an MI-consistent dialogue. The second set, assessing MI skills application, is a list of sixteen 6-point Likert-scale items allowing for self-measurement of the frequency of MI-adherent and non-adherent skills and behaviors in the participant’s clinical practice. For each of the sixteen statements, participants rate the frequency of the behavior as: “Never”, “Very rarely”, “Rarely”, “Moderately”, “Frequently” and “Extremely”. The last 5 items of our written questionnaire cover self-confidence in using MI in clinical practice. Each of the five consist of a 10-point Likert-scale ranging from 1, meaning a low level of self-confidence, to 10 meaning high level of self-confidence.
Coding protocol for the open-response question (adapted from HRQ)
Two items inspired from the HRQ were included into the questionnaire in order to assess the ability to apply MI-related skills.Citation37 Items are a vaccination-specific situational open-response question that participants are instructed to answer by writing a MI-consistent real-life dialogue. In order to code the question, a coding manual was developed, detailing the coding procedure. This coding manual was adapted from the MITI coding manual version 4.2.Citation27 Briefly, our coding manual comprises two components: 1- scores for four MI skills and 2- a global score of MI-spirit called “adherent score”. The coder must identify and count as MI-related skills each open-ended question (1 point), reflective-listening statement (between 1 and 2 points depending on complexity level), affirmative statement (1 point) and “elicit – provide – elicit” feature (between 1 and 3 points depending on the completion status). The answer to this open-response question must be in the form of a dialogue, otherwise no point is awarded. For the adherent score, the evaluator must appreciate the dialogue as a whole, coding occurs only on vaccinator interventions. Adherent score is evaluated on a scale from 1 to 5, with 3 represent the score to start from. MI-inconsistent behaviors perceived in the written dialogue, shift the adherent score closer to 1. On the other hand, where the respondent respects the MI-spirit, adherent score shifts closer to 5.
Content and face validity
In order to assess the content and face validity of the questionnaire, we contacted four local and international MI experts, who are members of the MINT. These experts were asked to identify the non-relevant and most relevant items of the questionnaire, as well as reformulate any questions needing clarification in their opinion. They were also instructed to identify any relevant items that we may have overlooked. The research team then deleted all items deemed as non relevant by the MI experts, as well as modified and wrote in new items according to the expert panel’s suggestions.
Participants and training protocol
After its content and face validity were assessed by the MI experts, the questionnaire was completed by nursing health professionals providing vaccination in health facilities, homes, schools and/or communities across the Eastern Townships (Quebec, Canada). The vaccination-specific MI training protocol consisted of two in-person training workshops: a one-day workshop followed by a half-day workshop three months later. The contents of the training provided an overview of MI theory and principles, followed by a practical demonstration and exercise to develop vaccination-specific MI-related skills. Training was provided by an MI expert, member of the MINT. Using a multiple pretest-posttest design, trainees were asked to complete the questionnaire before and after each day of training. Pre-test questionnaires were sent out by email two weeks prior to training and participants were invited to return their completed questionnaire either by email, by fax, or in person on the morning of the first day of training. Post-test questionnaires were sent out by email two days after the training and participants were given two weeks to complete and return it back by email or fax.
Intra- and inter-coder reliability
Reliability of the coding for the written open-response item was assessed. Coding of this item was first performed on a small random selection of questionnaires (20%) by a pair of MI experts, members of the MINT. Once good inter-coder reliability was established, the remainder of the questionnaires were coded by a single MI expert. In order to assess intra-coder reliability, the same MI expert randomly coded 20% of the questionnaires two weeks later.
Data analysis
MISI scoring
In the MI-knowledge area, multiple-choice questions were dichotomously coded, with 1 point allotted for each correct answer and 0 for any wrong answers. A dichotomous analysis was also performed for the 6-point Likert-scale items from the MI-skills area. For the MI-adherent items, the two highest categories of responses (“Frequently” and “Extremely”) were grouped and attributed 1 point. MI non-adherent items, were each attributed 1 point but only for the lowest category of responses (“Never”). Continuous variables in the third area – self-confidence in using MI-skills in clinical practice – were coded according to the number of points scored in the 10-point Likert-scale items. The totals for each area were then calculated by summing each item contained in the respective area. Raw totals were converted to 0–100 scales using simple linear transformation, accounting for items with missing values. Data from sections of the questionnaire with over 20% unanswered items were excluded from analyses. For the open-response question, the total score was calculated by summing the total points attributed for counts of each of the four MI skills and the score of the MI-Adherent item.
Calculated scores are presented using mean and standard deviation (SD). All statistical analyses were performed using SPSS version 24.
Internal consistency
In order to verify whether items measure the same general construct, analysis of internal consistency of domains was performed using the Kuder-Richardson coefficient (KR20) and Cronbach’s α for dichotomous and continuous variables, respectively.Citation40
Reliability
Test-retest reliability was measured for the MI-knowledge area by determining the concordance between mean scores at post-training day 1 and then two months later using the ICC. Mean scores were also compared using paired Student’s t-test for detecting significant difference between both scores.
The ICC was also used to determine intra- and inter-coder reliability of scores on the open-response question.Citation41
Effect size
Finally, questionnaire responsiveness was analyzed by comparing data measured prior to the first day of training and 3-months later following the half-day training workshop. Mean scores were compared using paired Student’s t-test and effect size (Cohen’s d) was calculated for each area.Citation42
Abbreviations
| BECCI | = | Behaviour Change counseling Index |
| HRQ | = | Helpful Response Questionnaire |
| ICC | = | intra-class correlation coefficient |
| MI | = | Motivational interviewing |
| MIKAT | = | Motivational interviewing knowledge and attitude test |
| MINT | = | Motivational Interviewing Network of Trainers |
| MISC | = | Motivational Interviewing Skills Coding System |
| MISI | = | Motivational Interviewing Skills in Immunization |
| MISTS | = | Motivational Interviewing Supervision and Training Scale |
| MITI | = | Motivational Interviewing Treatment Integrity Code |
| SD | = | Standard deviation |
Disclosure of potential conflicts of interest
No potential conflict of interest was reported by the authors.
Supplemental Material
Download MS Word (140 KB)Acknowledgments
We thank all of the nursing health professionals who participated in this study as well as all of the experts who provided their generous help in developing the questionnaire, including the MINT experts who reviewed the questionnaire, Danielle Pinsonneault, Karine Gaudreault, Jacques Dumont and Patrick Berthiaume. We also wish to thank Marie-Claude Battista and Monique Sullivan for critical revision of this manuscript.
Supplementary material
Supplemental data for this article can be accessed on the publisher’s website.
Additional information
Funding
Related Research Data
References
- World Health Organization. Meeting of the strategic advisory group of experts on immunization, April 2013 – conclusions and recommendations; 2013: 88(20):201–6.
- Sadaf A, Richards JL, Glanz J, Salmon DA, Omer SB. A systematic review of interventions for reducing parental vaccine refusal and vaccine hesitancy. Vaccine. 2013;31:4293–304. doi:10.1016/j.vaccine.2013.07.013.
- Napolitano F, D’Alessandro A, Angelillo IF. Investigating Italian parents’ vaccine hesitancy: A cross-sectional survey. Hum Vaccin Immunother. 2018;14:1558–65. doi:10.1080/21645515.2018.1463943.
- Dubé E, Vivion M, Sauvageau C, Gagneur A, Gagnon R, Guay M. Nature does things well, why should we interfere?”: vaccine hesitancy among mothers. Qual Health Res. 2016;26:411–25. doi:10.1177/1049732315573207.
- Wheeler M, Buttenheim AM. Parental vaccine concerns, information source, and choice of alternative immunization schedules. Hum Vaccin Immunother. 2013;9:1782–89. doi:10.4161/hv.25959.
- Dubé E, Gagnon D, MacDonald NE. SAGE working group on vaccine hesitancy, strategies intended to address vaccine hesitancy: review of published reviews. Vaccine. 2015;33:4191–203. doi:10.1016/j.vaccine.2015.04.041.
- MacDonald NE. SAGE working group on vaccine hesitancy, vaccine hesitancy: definition, scope and determinants. Vaccine. 2015;33:4161–64. doi:10.1016/j.vaccine.2015.04.036.
- Kaufman J, Ryan R, Walsh L, Horey D, Leask J, Robinson P, Hill S. Face-to-face interventions for informing or educating parents about early childhood vaccination. Cochrane Database Syst Rev. 2018;5:CD010038. doi:10.1002/14651858.CD010038.pub3.
- Gagneur A, Lemaître T, Gosselin V, Farrands A, Carrier N, Petit G, Valiquette L, De Wals P. A postpartum vaccination promotion intervention using motivational interviewing techniques improves short-term vaccine coverage: promoVac study. BMC Public Health. 2018;18:811. doi:10.1186/s12889-018-5724-y.
- Gagneur A, Petit G, Valiquette L, De Wals P. An innovative promotion of vaccination in maternity ward can improve childhood vaccination coverage. Report of the Promovac study in the Eastern Townships. [in French] Library and National Archives of Canada 2013. ISBN: 978–2–9813830–1–3
- Gagneur A, Gosselin V, Dubé E. Motivational interviewing : a promising tools to adress vaccine hesitancy? Vaccine. 2018;36:6553–55. doi:10.1016/j.vaccine.2017.10.049.
- Gagneur A, Lemaître T, Gosselin V, Farrands A, Carrier N, Petit G, Valiquette L, De Wals P. Promoting vaccination at birth using motivational interviewing techniques improves vaccine intention. J Infect Dis Ther. 2018;6. doi:10.4172/2332-0877.1000379.
- Appiah-Brempong E, Okyere P, Owusu-Addo E, Cross R. Motivational interviewing interventions and alcohol abuse among college students: a systematic review. Am J Health Promot. 2014;29:e32–42. doi:10.4278/ajhp.130502-LIT-222.
- DiClemente CC, Corno CM, Graydon MM, Wiprovnick AE, Knoblach DJ. Motivational interviewing, enhancement, and brief interventions over the last decade: A review of reviews of efficacy and effectiveness. Psychol Addict Behav. 2017;31:862–87. doi:10.1037/adb0000318.
- Davis MM, Gance-Cleveland B, Hassink S, Johnson R, Paradis G, Resnicow K. Recommendations for prevention of childhood obesity. Pediatrics. 2007;120(Suppl 4):S229–253. doi:10.1542/peds.2007-2329E.
- Lindson-Hawley N, Thompson TP, Begh R. Motivational interviewing for smoking cessation. Cochrane Database Syst Rev. 2015:CD006936. doi:10.1002/14651858.CD006936.pub3.
- Rubak S, Sandbaek A, Lauritzen T, Christensen B. Motivational interviewing: a systematic review and meta-analysis. Br J Gen Pract. 2005;55:305–12.
- Kuyper L, de Wit J, Heijman T, Fennema H, van Bergen J, Vanwesenbeeck I. Influencing risk behavior of sexually transmitted infection clinic visitors: efficacy of a new methodology of motivational preventive counseling. AIDS Patient Care STDS. 2009;23:423–31. doi:10.1089/apc.2008.0144.
- A.F. Dempsey, K. Garrett, S. Lockhart. Impact of MI training on provider’s communication about adolescent HPV vaccination. Paper presented at Pediatric Academic Societies Meeting 2016; 2016; Baltimore.
- Dempsey AF, O’Leary ST. Human Papillomavirus vaccination: narrative review of studies on how providers’ vaccine communication affects attitudes and uptake. Acad Pediatr. 2018;18:S23–S27. doi:10.1016/j.acap.2017.09.001.
- Dempsey AF, Pyrznawoski J, Lockhart S, Barnard J, Campagna EJ, Garrett K, Fisher A, Dickinson LM, O’Leary ST. Effect of a health care professional communication training intervention on adolescent human Papillomavirus vaccination: a cluster randomized clinical trial. JAMA Pediatr. 2018;172:e180016. doi:10.1001/jamapediatrics.2018.0016.
- Miller WR, Rollnick S. Motivational interviewing: preparing people to change addictive behavior. 2nd ed. New York (NY): Guilford; 2002.
- Miller WR, Rollnick S. Motivational interviewing: preparing people to change addictive behavior. New York (NY): Guilford; 1991.
- Rollnick S, Miller WR, Butler CC. Motivational interviewing in health care: helping patients change behavior. New York (NY): Guilford Press; 2008.
- Söderlund LL, Madson MB, Rubak S, Nilsen P. A systematic review of motivational interviewing training for general health care practitioners. Patient Educ Couns. 2011;84:16–26. doi:10.1016/j.pec.2010.06.025.
- Madson MB, Loignon AC, Lane C. Training in motivational interviewing: a systematic review. J Subst Abuse Treat. 2009;36:101–09. doi:10.1016/j.jsat.2008.05.005.
- Moyers TB, Rowell LN, Manuel JK, Ernst D, Houck JM. The motivational interviewing treatment integrity code (MITI 4): rationale, preliminary reliability and validity. J Subst Abuse Treat. 2016;65:36–42. doi:10.1016/j.jsat.2016.01.001.
- Pierson HM, Hayes SC, Gifford EV, Roget N, Padilla M, Bissett R, Berry K, Kohlenberg B, Rhode R, Fisher G. An examination of the motivational interviewing treatment integrity code. J Subst Abuse Treat. 2007;32:11–17. doi:10.1016/j.jsat.2006.07.001.
- Moyers TB, Martin T, Manuel JK, Hendrickson SML, Miller WR. Assessing competence in the use of motivational interviewing. J Subst Abuse Treat. 2005;28:19–26. doi:10.1016/j.jsat.2004.11.001.
- Lane C, Huws-Thomas M, Hood K, Rollnick S, Edwards K, Robling M. Measuring adaptations of motivational interviewing: the development and validation of the behavior change counseling index (BECCI). Patient Educ Couns. 2005;56:166–73. doi:10.1016/j.pec.2004.01.003.
- Miller WR. Motivational interviewing skill code (MISC): Coder’s manual. University of New Mexico, Center on Alcoholism, Substance Abuse and Addictions (CASAA); 2000.
- Madson MB, Campbell TC, Barrett DE, Brondino MJ, Melchert TP. Development of the motivational interviewing supervision and training scale. Psychol Addict Behav. 2005;19:303–10. doi:10.1037/0893-164X.19.3.303.
- Gibbons CJ, Carroll KM, Ball SA, Nich C, Frankforter TL, Martino S. Community program therapist adherence and competence in a motivational interviewing assessment intake session. Am J Drug Alcohol Abuse. 2010;36:342–49. doi:10.3109/00952990.2010.500437.
- Leffingwell TR. Motivational interviewing knowledge and attitudes test (MIKAT) for evaluation of training outcomes. Minuet. 2006;13:10–11.
- Hartzler B. Adapting the helpful responses questionnaire to assess communication skills involved in delivering contingency management: preliminary psychometrics. J Subst Abuse Treat. 2015;55:52–57. doi:10.1016/j.jsat.2015.02.006.
- Baer JS, Rosengren DB, Dunn CW, Wells EA, Ogle RL, Hartzler B. An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug Alcohol Depend. 2004;73:99–106. doi:10.1016/j.drugalcdep.2003.10.001.
- Miller WR, Hedrick KE, Orlofsky DR. The helpful responses questionnaire: a procedure for measuring therapeutic empathy. J Clin Psychol. 1991;47:444–48. doi:10.1002/(ISSN)1097-4679.
- Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs: the four levels. San Francisco (CA): Berrett-Koehler; 2006. [ accessed 2018 Aug 31. http://www.aspresolver.com/aspresolver.asp?BIZP;2341903.
- Cucciare MA, Ketroser N, Wilbourne P, Midboe AM, Cronkite R, Berg-Smith SM, Chardos J. Teaching motivational interviewing to primary care staff in the veterans health administration. J Gen Intern Med. 2012;27:953–61. doi:10.1007/s11606-012-2016-6.
- Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess. 2003;80:99–103. doi:10.1207/S15327752JPA8001_18.
- Fleiss JL. Chapter one, in: design and analysis of clinical experiments. New York (NY): Wiley Classis Library; 1999.
- Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Lawrence Erlbaum Associates, New York; 1988.
- McDowell I. Measuring health: a guide to rating scales and questionnaires. 3rd ed. Oxford; New York: Oxford University Press; 2006.
- Shafer MS, Rhode R, Chong J. Using distance education to promote the transfer of motivational interviewing skills among behavioral health professionals. J Subst Abuse Treat. 2004;26:141–48. doi:10.1016/S0740-5472(03)00167-3.