1,941
Views
0
CrossRef citations to date
0
Altmetric
Articles

How to integrate addiction medicine in psychiatry training: results of an experiment with two educational methods

ORCID Icon, ORCID Icon, ORCID Icon, ORCID Icon, ORCID Icon & ORCID Icon
Pages 1-11 | Received 11 Oct 2018, Accepted 08 Dec 2018, Published online: 01 Jan 2019

ABSTRACT

The high rates of comorbidity with substance use disorders in general psychiatry patients demand enhanced competences from psychiatry residents in addiction medicine. The aim of this article is to improve knowledge, skills and attitudes in psychiatric residents in treating patients with comorbid substance-related disorders (SUD). Four seminars with all residents on relevant and actual knowledge on substance-related disorders and a small group experience of five sessions on improving skills and attitudes in dealing with patients are described and evaluated quantitatively and qualitatively. Both quantitative evaluations show that seminars and small group experiences are highly appreciated. Concerning the seminar, residents appreciated, in particular, to study and present selected up-to-date literature and the positive learning environment. The small group experience resulted in sharing feelings and thoughts about patients with SUD and mutual support. The results show that the goals of these two methods of education have been met. This means that these two methods can be integrated in the training of residents in psychiatry to increase knowledge, skills and attitudes concerning addiction.

Introduction

As in most western countries, addiction is a major healthcare problem in Lithuania. It is estimated that in Lithuania between 8% and 19.8% (Veryga, Stanislavovienė, Ignatavičiūtė, Štelemėkas, & Liutkutė, Citation2013) of the population is using alcohol excessively. On average, a person of 15 years and older consumed 14.0 l of alcohol per person in 2015 that was attributable to 25.2 alcohol-related mortality per 100,000 (Narkotikų tiakd, Citation2015).

The burden of potential life lost in 2010 was 842/100,000 (Štelemėkas, Jaselskytė, Liutkutė, & Veryga, Citation2013). In 2014, nearly 22 of every 100 Lithuanian inhabitants aged 15 years or older (21.5%) smokers (Oficialiosios statistikos p, Citation2015). Lifetime prevalence of illegal substances use was 11.1 in 2012 (EMCDDA, Citation2016). Most of the patients affected with SUDs are treated by multidisciplinary teams in community mental health care centres. More complex needs of the patients are addressed by the specialists at the regional subsidiaries of the Republican Centre for Addictive Disorders.

In all European countries (Laramee et al., Citation2013), alcohol-/ substance-related problems are a burden for the persons themselves, their families and the society (Effertz & Mann, Citation2013; Whiteford et al., Citation2013). Until recently, addiction medicine was not well integrated in undergraduate and postgraduate medical curricula (Ayu, Schellekens, Iskandar, Pinxten, & De Jong, Citation2015; Carroll et al., Citation2014). Internationally, there are initiatives to improve competencies in medical doctors to treat patients with substance-related disorders (De Jong et al., Citation2016).

One of the recent initiatives was that the Radboud University Nijmegen invited, in 2014, an international group of Addiction Medicine (AM) physicians and educational specialists. They identified barriers to the introduction of AM training, shared best practices in order to develop a comprehensive educational model, and brought together existing initiatives on improving such trainings. The meeting on addiction medicine training resulted in the following consensus statements. Excessive substance use and addictive behaviours are common, associated with different medical problems, and a great burden for the individual and the society (a), patients with excessive substance use and addictive behaviours are insufficiently identified and receive inadequate care (b), negative perceptions and attitudes towards addicted patients are common among health professionals, including medical doctors (c) and training in Addiction Medicine can improve knowledge, attitudes and clinical practice related to addiction and addictive behaviours (d). The group concluded that it is in the best interest of patients and health policy to make addiction medicine interesting for young physicians. Therefore, universities should develop an AM curriculum to make physicians competent in all levels to work with patients with an addiction. It is also acknowledged that the treatment of somatic and psychiatric disorders has to be improved if a patient is also suffering from a substance-related disorder. It is concluded that concerning training in different countries, there is no ‘one size fits all model'. Some countries focus on the primary care; others focus on specialties including psychiatry, internal medicine and community health; and most are adopting a dual-policy approach.

In recent years, much attention is paid to research-based or research-informed learning in medical education. Although the seminal work of Edgar Dale (Dale, Citation1969) on learning is frequently cited (‘Direct, first hand experiences that make up the foundations of our learning'), it is acknowledged that research on medical education should apply the same standards and expectations of quality to educational research that is expected in clinical trials (Masters, Citation2013). Education can and should also be measured qualitatively (Roman, Citation2014) and the research shows that blending teaching methods contributes to learning retention and student’s overall satisfaction on learning (Sadeghi, Sedaghat, & Sha Ahmadi, Citation2014).

Considerable amount of research has focused attention on the relation between attitude and behaviour change, namely, how attitude change corresponds to actual behaviour, e.g. gaining knowledge and skills and putting it into practice (Ajzen & Fishbein, Citation1977).

Addicted patients are treated by the psychiatry professionals in Lithuania. Medical doctors get specialization in psychiatry during a four-year residency training at the university psychiatry clinic. Since 2012, the training has included a three-month supervised practice in addiction psychiatry at the specialized addiction treatment centre. The course is accompanied by lectures and seminars on relevant topics in addiction.

Although addiction medicine becomes more and more integrated in the curriculum of the training of psychiatric residents in Lithuania, there is a strong need for more competent doctors to cope with the large number of addicted patients. In 2016, we therefore started with a knowledge, skills and attitude enhancement project with all residents of the psychiatry department of the Lithuanian University of Health Sciences Kaunas. We used the educational principles used in the Dutch Addiction Medicine Specialist course (De Jong & Luycks, Citation2015) for a seminar for all residents on knowledge as presented in recent articles on addiction and general concepts of group dynamics (Yalom & Leszcz, Citation2005) for exploring attitudes in a small group experience with the first year residents.

In this article, the educational project of Lithuanian residents in psychiatry is described in detail as are the educational methods. The aim of the project was to improve knowledge, skills and attitudes in psychiatric residents in treating patients with comorbid substance-related disorders (SUD). Although we realize that the most important thing would be to study the effect of such a project on the competencies of residents in dealing with patients, we decide to start evaluating the feasibility and acceptance by the residents attending the session of the project. Therefore, a standardized evaluation was part of the project and the results are described in a quantitative and qualitative way and will lead to recommendations for integration of addiction medicine in the training of future psychiatrists and recommendations for educational methods to do so.

Materials and methods

Participants

Residents of all four years of psychiatry training (n = 52) participated in the seminar, mean age 29.2 (4.5), 41 women and 11 men. The small group consisted of the first year residents only (n = 11), mean age 29.0 (sd 7.7), 10 women and 1 man. We thought that the first group would profit most by the experiential approach because they were at the start of their training.

There were two moderators. The first moderator (CDJ) is a professor in addiction and addiction care affiliated to the Radboud University in the Netherlands. He was trained as a medical doctor and psychotherapist and he has been working in the field for more than 40 years and is the principal lecturer of Dutch Addiction Medicine Specialist training course. The second moderator (DJ) has been a psychiatrist working in the field of addiction psychiatry for more than 20 years. He supervises the residents from the third and fourth year when they are dealing in addiction psychiatry.

Knowledge, skills and attitude enhancement

SEMINAR: Concerning knowledge enhancement, the residents had to read recent articles on the definition of addiction, assessment and diagnosis of substance-related disorders, epidemiology, burden of the disease, stigmatization, pathophysiology including genetics and neurobiology, treatment including recovery and psychiatric comorbidity (see for the content of papers).

Table 1. Reader for the workshop on addiction and psychiatry.

They were asked to present their critical appraisals of the articles during plenary sessions. In a detailed guideline, it was described how they should read the articles, summarize them, appraise them in a critical way and to present them in front of the plenary group (the guideline can be sent to the readers by the first author). They were also provided with a short list of tips for presentations (Canadian-Cochrane-Centre, Citation2005). Communication is one of the basic competencies of medical doctors, including presentations for colleagues. So, improving skills to present scientific information was one of the goals of the seminar project. The quality of their presentation was assessed by means of a standardized form addressing the content, the design, verbal and non-verbal presenting and the use of audiovisual aids. For every domain, a formative assessment was given with a Likert scale ranging from Very Good, Good, Acceptable, Poor and Very Poor. The score could range from 4 to 20. (The form can be sent to the readers by the first author).

At the end of every presentation, the audience was invited to come up with questions and comments. The moderator invited the presenter to assess his or her presentation with two main questions: ‘What went well?’ and ‘What could be improved?’ After that the audience could give their comments, first positive feedback and then suggestions for improvement. The moderator wrapped up the presentation and added personal ideas or evidence-based information.

SMALL GROUP EXPERIENCE: We strongly believe that it is essential to work on skills and attitudes for doctors who work with addicted patients. One of the skills is to learn how to deal with the complex feelings these patients evoke in care givers (Forrest, Citation2002). Usually, there are negative thoughts, feelings or reactions that confirm the stigma that these patients carry with them. The first step in improving the skills is to explore the attitudes of care givers. So, we started a small group experience with the first year residents. In the group, we shared their experiences with addiction or problematic use of substances. It could concern patients, someone outside the consulting room, someone close to them or even themselves. The opening question was: ‘What do you think, feel and do in such occasions?’.

Evaluation procedure

Because of the rather unique character of the two educational methods, we were not able to find validated questionnaires for them. So, an evaluation form was developed with open questions and items to be rated on a Likert scale. The questions and items in the form were based on the aspects of the small group and the seminars. The form was the same for both groups. For the small group, two questions were added concerning the personal involvement of the moderators. At the end of each forms, the residents were asked to give score for the total seminar on a Visual Analogue Scale (VAS) ranging from 1 (very bad) to 10 (very good). For the evaluation of the seminar, two questions were added: ‘Regarding the literature for the seminar I am very dissatisfied (1) to very satisfied (10)’ and ‘Regarding the way the moderator dealt with the group and me I am very dissatisfied (1) to very satisfied (10)’.

The forms were handed to the residents at the end of the last session and were filled in preferably on the spot. Afterwards, the filled-in forms were handed back to the moderators and it was guaranteed that the data remained anonymous after that the quantitative data were entered in an SPPS file and the qualitative data in a Word file. Descriptive statistics were calculated for the quantitative data. The answers on the open questions were analysed by two moderators and one of the participating residents for each of the groups, with the following instruction:

Please take a look at the answers and try to classify them in a few categories. Beyond these main categories you may find important details. Write down your conclusion(s) and share the outcome in a meeting in which you will have to come to a final conclusion for each of the answers.

Results

Here we present the results to evaluate the feasibility and acceptance by the residents. shows the results of the quantitative analysis of both methods (small group and seminar group) and comparison between them (student t-test). Though participants of the small group gave higher scores, a statistically significant difference was in only one statement ‘I learned a lot about myself and am ready to make definite changes'. The small group gave higher score compared to the seminar group (t = −2.505, p < .05).

Table 2. Opinions about the two educational methods (Using a scale of 5 being the highest to 1 the lowest), except for the last three items (1–10)

Small group experience

All residents filled in the evaluation form. The qualitative analysis resulted in a summary of all open-ended questions as follows: Before the group started, the residents wanted to learn more about understanding and helping patients and about feelings arising in the group (A). After the group, they noted that they started caring more about the feelings of their colleagues, patients’ feelings and their own feelings (B). Additionally, it was mentioned that communication in Lithuanian instead of English would have made it easier to share feelings. Some of them were ambivalent about the role of the Lithuanian moderator (C). The most useful things students learned were being more open with their own feelings, recognize them and getting positive feedback (1). They could practise more on listening, sharing and coping with their own feelings (2) and endorse participating in the group, their expectations were exceeded, although not everyone had predefined expectations (3). After the group, the residents felt closer together, understood their feelings better (4) and admitted that they made progress towards sharing feelings among them (5). For future groups, most of them were strongly advised to attend the group and to share their feelings in the group (6). In a sense, they had the liberty to talk with the moderator from abroad and expressed themselves more freely (8). However, they felt less comfortable in front of the Lithuanian moderator who is affiliated to the staff of their training (9). They contributed to the high ranking of the whole experience in the group setting, atmosphere in the group, personality and professionalism of the moderators and the opportunity for personal involvement (11). The group could be improved by more and longer sessions.

Seminar

Thirty-six of the 52 residents filled in the evaluation form (69%). shows the results of the quantitative analysis of both methods. In general, the scores for the seminar are also fairly high, ranging from 3.11 (I learned a lot about myself and am ready to make definite changes) to 4.58 (If other residents ask me if they should participate in a similar group, I would recommend that they ‘give-it-a-try').

The qualitative analysis resulted in a summary of all open-ended questions as follows: most of residents wanted to learn about pathophysiology and treatment of SUDs. (A) After the seminar, most of the residents felt more compassionate about people with SUDs and understood the importance of keeping up with the updated knowledge (B) Residents enjoyed studying in seminars and especially remarked well about the positive learning environment (C). Most residents said that they deepened their knowledge in the treatment of patients with SUD. They also improved their presentation skills and they appreciated especially the positive learning environment (1) A lot of residents wanted to delve deeper into pathophysiology of SUDs and practice on presentation skills (2). Most of the expectations of residents were exceeded. Although some wished that moderators would have shared more of their own experience (3), residents felt more empowered in the treatment of SUDs, they experienced what a positive learning environment could be like (4). Most were insecure about their competencies in the treatment of patients. Some gained more confidence or felt that they improved in diagnosing and treating SUDs (5). Quite a few residents wanted to hear more of moderators’ own experience. Some of them wanted to work more in small groups, some mentioned that their supervisors should come too, some became more eager to work with SUD cases (6). Most of the residents mentioned that some parts of the literature were hard to understand (7). All the residents loved the sincerity of the moderators and the culture of communication (8). Most residents prioritized feelings and emotions (9). General consensus was that there is almost no way to do it better. Although some said that there should have been more active involvement of moderators, residents wanted more in-depth opinions on various topics from them (11).

Quality of presentations

The papers in the reader were presented by 24 residents. The quality of their presentations ranged from 9 to 17 on a scale from 4 to 20, with a mean of 11.8 (sd 2.34). All residents presented their critical appraisal of the papers in English, which of course is not their native language. So, they all deserve a compliment for doing so. In general, they were able to summarize the paper quite well and they used slides according to the guidelines provided by the moderators. However, there are several recommendations for the improvement of their skills in scientific presenting. For instance, start with the answer on the question why/what you are going to present and give information about the content. In a presentation, the oral and visual aspects should be additional. So, do not read what is on the slide but develop your thoughts on the topic that is visible. You are expected to critically appraise an article, so you are allowed to be critical, of course in a friendly way. Use nice pictures, but do not overwhelm your audience with too much of them. If you use graphs, explain what can be seen with a laser pointer. Make use of a remote control to change slides, otherwise you will stick to the lecture. Rehearse your presentation with some friendly but critical colleagues. If you want to use crib sheets, please put page numbers on them.

Comparison of ratings from the attendants of the small group and the seminar

Concerning the qualitative evaluation, there is a considerable overlap in appreciation, although there are differences. There are certain difficulties comparing response from the small and the big group as the learning experience was quite different. However, after the training, respondents from both groups expressed better understanding of and empathy towards the patients suffering from SUDs. Trainees from the small group noted that they had the opportunity to understand the feelings of the patients, learned to share it in the group and express mutual support. As the big group increased the knowledge in pathophysiology, diagnostics and treatment perspectives that in a way helped to understand the patients better as well.

Discussion

Regarding the quantitative results, almost all small group score evaluations were higher compared to those of the seminar group, although they were not statistically different. Residents in the small group gave highest scores when they evaluated working with other residents and moderators in the group. The highest score was for the item: ‘If other residents ask me if they should participate in a similar group, I would recommend that they “give-it-a-try”’. But, residents in the seminar group gave higher scores compared to those in the small group when they were asked to evaluate new skills and willing to use them in their practices. Both groups gave very high scores for the total evaluation. In a scale from 1 to 10, the small group got 9.18 (sd: 0.98) and the seminar group got – 8.72 (sd: 0.98) points. Sadeghi et al. (Citation2014) compared the effect of lectures and blended teaching methods on students’ learning and satisfaction and found rather similar results that a blended method is effective in increasing the students’ learning rate. The high appreciation of these methods could also reflect their novelty and an increased motivation towards new methods.

Talking about qualitative results, before the seminar, most of residents indicated that they expect to learn more about the pathophysiology and treatment of SUDs. After the seminar, we noted that most of the residents indicated that they felt more compassionate about people with SUDs and understood the importance of keeping up with updated knowledge. This shift could mean that residents’ view moved from a technical perspective to a more compassionate/empathic understanding of patients. Most residents said that they deepened their knowledge in the treatment of SUDs and they especially liked the positive learning environment, which was mentioned quite a few times in their answers. A lot of residents wanted to delve deeper into the pathophysiology of SUDs. This could mean that residents did not have enough material to read or they did not have enough background knowledge to comprehend the material or the material was not prepared too well, since residents were keen to learn pathophysiology before the seminar. Furthermore, the residents wanted to practise on presentation skills, which was not included explicitly in this course curriculum. Although most of the residents’ expectations were exceeded, some wished that moderators would have shared more of their own experience. After the seminar, residents felt more empowered in the treatment of SUDs. They were quite happy to find out how to work in a positive and safe learning environment. Most were unsure as to what progress they have made towards goals set by themselves, which could mean that they either forgot what goals they have set before the seminar, or they did not set any goals or they have not met goals, but felt enthusiastic. Some gained more confidence or felt that they got better in diagnosing and treating SUDs. Quite a few residents recommended the moderators in future seminars to share more of their own experience. Some of them wanted to work more in small groups, some mentioned that our supervisors should come too, some were eager to work with individual cases. Most of the residents mentioned that some parts of the literature were hard to understand. What could be caused by the lack of knowledge in English language or lack of background knowledge in addiction medicine. All the residents loved the sincerity of moderators and the way they communicated with them. A note was made that some felt discomfort to share experiences in a small group in the presence of a moderator who was affiliated to the training staff of the clinic. That confirmed the rule that a qualified specialist from outside the clinic should be invited for this part of work. When residents gave rankings for the seminar, most residents gave priority to feelings and emotions. Which is also probably due to the positive learning environment. General consensus was that there is almost no way to do it better. Although some said that there should have been more active involvement of moderators that residents wanted to get.

The quality of presentations was in the range from 9 to 17 with a mean of 11.8 and no presentation was scored 20. That indicates an average quality of presentations and a need to improve presentation skills. Scoring itself brings a sense of competition and serves as a motivational factor.

However, this study has certain limitations such as an absence of evaluation before the beginning of seminars and small group and there were no control groups. Both groups were rather small and this could explain why there were statistically significant differences between them. Concerning the qualitative part of the evaluation, we did not analyse the data based on the grounded theory approach, but in doing what we did we could categorize the comments in a more or less structured way.

All the first-year residents evaluated the small group experience. The seminar was evaluated by 69% of the residents. The remaining 31% consisted of residents that could not attend the last session, because they were already on leave for the weekend, had to finalize their weekly work or were on duty. We have no reasons to believe that they did not come to the presentation session because they were evaluating the seminar in a negative way and would not be willing to fill in the evaluation form.

The study did not address the difference between self-assessment and actual behaviour (Davis et al., Citation2006) as on this first occasion the intension was to evaluate also the appreciation of educational methods. It proved to be hopeful and the next step would be to measure the effect of consolidated educational plan on the behaviour towards patients with addiction.

Conclusions and recommendations

Both groups appreciated the experience and considered it to be useful for emphatic understanding of the patient and further learning of clinical skills. It brought more knowledge on SUDs, more caring about patients and improvement of presentation skills. Both learning methods had a different positive impact and proved to be additional. As far as we know, this is the first study on the feasibility and acceptance of two educational methods not regularly included in curricula for psychiatrists in training.

Although we did not perform an efficacy or effectiveness study on the two educational methods, we dare to recommend that formatively assessed seminars and small group methods, similar to the ones we described and evaluated, here become integrated in the training of residents in psychiatry to increase knowledge, skills and attitudes concerning addiction and presentation skills.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Darius Jokubonis, MD, is education assistant at the Psychiatry Department of Lithuanian University of Health Sciences, Lithuania. He was trained as a medical doctor and completed the residency program in psychiatry and specialized in addiction psychiatry during forthcoming years. Since 2013 he is in charge of training of psychiatry residents in addiction medicine.

Prof. Virginija Adomaitiene is a head of the Psychiatry Department, Lithuanian University of Health Sciences, Lithuania. Research interests include integration of psychiatry in general medicine, timely recognition, diagnosis and treatment of mental disorders in patients with somatic diseases. A member of the board of Lithuanian Association of Psychiatrists. Since 2012 a member of WHO working group in psychiatry (European region).

Laura Baltaityte is a 4th year resident doctor in adult psychiatry at Vilnius University, Lithuania. Also a board member of Lithuanian Art Therapy Association and a psychoanalytic group psychotherapy studies trainee at Vilnius University, Lithuania. She is working individually as an art therapist and leads a psychodynamic group in a private setting. Interested in addiction medicine in clinical setting and searching new ways to treat addiction and as a resident doctor searching for better ways to integrate studying into group experience.

Edgaras Dirzius, MD, is a PhD student at Lithuanian University of Health Sciences (LUHS), Institute of Biology Systems and Genetic Research, assistant lecturer at LUHS Psychiatry department and psychiatrist at Hospital of LUHS ‘Kaunas clinics’. He is a president of Junior Psychiatrist Association, vice president at Lithuanian Society for Biological Psychiatry.

Cornelis Aj De Jong, MD, PhD, is emeritus professor in addiction and addiction care at the faculty of social sciences of the Radboud University Nijmegen, the Netherlands. Until 2016 he was the scientific director of the Nijmegen Institute for Scientist-Practitioners in Addiction (NISPA). He is the author of more than 100 international papers in peer-reviewed journals. His research interests have always been the neurobiology of addiction and the comorbidity of substance-use disorders and other psychiatric disorders, especially personality disorders in relation to the interpersonal model of personality. Recently his research focuses on the chronicity of substance-use disorders, shared decision making, explanatory models of addiction in patients and professionals and the transcultural aspects of addiction. Until 2017 he was the principal lecturer of the Dutch Master in Addiction Medicine at the faculty of medicine of the Radboud University Nijmegen. He is one of the founders of the Dutch Physicians Health Program. He is co-chair of the Training and Education Committee of the International Society of Addiction Medicine (ISAM).

References

  • Ajzen, I., & Fishbein, M. (1977). Attitude-behavior relations: A theoretical analysis and review of empirical research. Psychological Bulletin, 84(5), 888–918. doi: 10.1037/0033-2909.84.5.888
  • Ayu, A. P., Schellekens, A. F., Iskandar, S., Pinxten, L., & De Jong, C. A. (2015). Effectiveness and organization of addiction medicine training across the globe. European Addiction Research, 21(5), 223–239. doi: 10.1159/000381671
  • Canadian-Cochrane-Centre. (2005). Tips for creating and using effective powerpoint presentations.
  • Carroll, J., Goodair, C., Chaytor, A., Notley, C., Ghodse, H., & Kopelman, P. (2014). Substance misuse teaching in undergraduate medical education. BMC Medical Education, 14, 34. doi: 10.1186/1472-6920-14-34
  • Dale, E. (1969). Audiovisual methods in teaching (3rd ed.). New York, NY: Holt, Reinhart & Winston.
  • Davis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA, 296(9), 1094–1102. doi: 10.1001/jama.296.9.1094
  • De Jong, C. A., Goodair, C., Crome, I., Jokubonis, D., El-Guebaly, N., Dom, G., et al. (2016). Substance misuse education for physicians: Why older people are important. Yale Journal of Biology and Medicine, 89(1), 97–103.
  • De Jong, C. A. J., & Luycks, L. (2015). Addiction medicine training in the Netherlands. In N. El-Guebaly, G. Carrà, & M. Galanter (Eds.), Textbook of addiction treatment: International perspectives (pp. 37–51). Milan Heidelberg New York Dordrecht London: Springer.
  • Effertz, T., & Mann, K. (2013). The burden and cost of disorders of the brain in Europe with the inclusion of harmful alcohol use and nicotine addiction. European Neuropsychopharmacology, 23(7), 742–748. doi: 10.1016/j.euroneuro.2012.07.010
  • EMCDDA. (2016). Statistical bulletin 2016. Retrieved from www.emcdda.europa.eu. http://www.emcdda.europa.eu/data/stats2016.
  • Forrest, G. G. (2002). Countertransference in chemical dependence counseling. New York, NY: The Haworth Press.
  • Laramee, P., Kusel, J., Leonard, S., Aubin, H. J., Francois, C., & Daeppen, J. B. (2013). The economic burden of alcohol dependence in Europe. Alcohol and Alcoholism, 48(3), 259–269. doi: 10.1093/alcalc/agt004
  • Masters, K. (2013). Edgar Dale’s Pyramid of learning in medical education: A literature review. Medical Teacher, 35(11), e1584–e1593. doi: 10.3109/0142159X.2013.800636
  • Narkotikų tiakd. (2015). Alkoholio vartojimas ir padariniai. Retrieved from http://ntakd.lrv.lt/lt/statistika-ir-tyrimai/alkoholio-vartojimas-ir-padariniai.
  • Oficialiosios statistikos p. (2015). Alkoholio ir tabako vartojimas ir padariniai 2014. Retrieved from https://osp.stat.gov.lt/informaciniai-pranesimai?eventId=62471.
  • Roman, I. (2014). Qualitative methods for determining students’ satisfaction with teaching quality. Procedia – Social and Behavioral Sciences, 149, 825–830. doi: 10.1016/j.sbspro.2014.08.320
  • Sadeghi, R., Sedaghat, M. M., & Sha Ahmadi, F. (2014). Comparison of the effect of lecture and blended teaching methods on students’ learning and satisfaction. Journal of Advances in Medical Education & Professionalism, 2(4), 146–150.
  • Štelemėkas, M., Jaselskytė, I., Liutkutė, V., & Veryga, A. (2013). Visiškai nuo alkoholio priklausančio ligotumo pokyčiai Lietuvoje 2006–2011 metais [Prevalence of fully alcohol attributable conditions in Lithuania 2006–2011]. Sveikatos Mokslai / Health Sciences, 23(5), 29–34. doi: 10.5200/sm-hs.2013.114
  • Veryga, A., Stanislavovienė, J., Ignatavičiūtė, L., Štelemėkas, M., & Liutkutė, V. (2013). Probleminio alkoholio vartojimo įvertinimas tarp 15–74 metų amžiaus Lietuvos gyventojų naudojant AUDIT metodiką [The analysis of problematic alcohol use among 15–74 years old Lithuanian population: Results of AUDIT test]. Sveikatos Mokslai / Health Sciences, 23(5), 22–28. doi: 10.5200/sm-hs.2013.113
  • Whiteford, H. A., Degenhardt, L., Rehm, J., Baxter, A. J., Ferrari, A. J., Erskine, H. E., et al. (2013). Global burden of disease attributable to mental and substance use disorders: Findings from the global burden of disease study 2010. Lancet, 382(9904), 1575–1586. doi: 10.1016/S0140-6736(13)61611-6
  • Yalom, I. D., & Leszcz, M. (2005). The theory and practice of group psychotherapy. New York, NY: Basic Books.