290
Views
6
CrossRef citations to date
0
Altmetric
Review

Gaming science innovations to integrate health systems science into medical education and practice

, &
Pages 407-414 | Published online: 31 May 2018

Abstract

Health systems science (HSS) is an emerging discipline addressing multiple, complex, interdependent variables that affect providers’ abilities to deliver patient care and influence population health. New perspectives and innovations are required as physician leaders and medical educators strive to accelerate changes in medical education and practice to meet the needs of evolving populations and systems. The purpose of this paper is to introduce gaming science as a lens to magnify HSS integration opportunities in the scope of medical education and practice. Evidence supports gaming science innovations as effective teaching and learning tools to promote learner engagement in scientific and systems thinking for decision making in complex scenarios. Valuable insights and lessons gained through the history of war games have resulted in strategic thinking to minimize risk and save lives. In health care, where decisions can affect patient and population outcomes, gaming science innovations have the potential to provide safe learning environments to practice crucial decision-making skills. Research of gaming science limitations, gaps, and strategies to maximize innovations to further advance HSS in medical education and practice is required. Gaming science holds promise to equip health care teams with HSS knowledge and skills required for transformative practice. The ultimate goals are to empower providers to work in complex systems to improve patient and population health outcomes and experiences, and to reduce costs and improve care team well-being.

Video abstract

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

http://youtu.be/3_JxRYBMw7U

Introduction

Global and national attention is on improving individual and population health. Population health initiatives focus on improving health outcomes and the distribution of outcomes in any given population.Citation1 Simple solutions to improve population health are non-existent. Socioeconomic status, environmental exposures, lifestyle behaviors, and access to care are examples of determinants that contribute to health outcomes and disparities.Citation2 The World Health Organization,Citation3 Institute for Healthcare Improvement (IHI),Citation4 and National Academy of MedicineCitation5 are organizations calling for systematic and community-based innovations to improve health systems and address population health.

Health systems require well-trained teams for delivery of quality services to effectively maximize health outcomes, minimize costs, and meet the needs of diverse populations.Citation3 Systems thinking, multidisciplinary collaboration, and innovative strategies are essential to prepare health professionals for roles in population health.Citation4Citation7 Physician leaders and medical educators are promoting health systems science (HSS) integration into education and clinical practice as a catalyst to improve population health.Citation8Citation11

HSS is an emerging discipline from the seminal work of the American Medical Association Accelerating Change in Medical Education Consortium, consisting of nearly one-fifth of all allopathic and osteopathic medical schools.Citation10 The Consortium is working to advance change in medical education to prepare physicians for the needs of individuals and populations now and in the future.Citation10 The provision of optimal care in complex systems requires an understanding of HSS, the applied science of how care is accessed, delivered, financed, managed, and assessed.Citation11 HSS conceptualizes how to work in collaborative teams for improved quality, value, and safety in the delivery of patient- and population-centric health services.Citation8Citation10 HSS aligns with the Quadruple Aim framework for improvement of population health, patient care experiences, work life of providers, and reduction of costs.Citation12 The study of HSS will give new physicians a broad view of the administrative challenges and societal factors that affect health outcomes and can complicate patient care. Knowledge of HSS can improve providers’ abilities to care for patients and populations.

The Consortium is promoting the field of HSS as the third pillar of medical education, along with the traditional pillars of biomedical and clinical sciences.Citation8Citation11 Yet, HSS integration into medical education and practice is not without challenges.Citation11 The difficulty of aligning HSS education with real-world scenarios related to complex delivery systems has been identified.Citation11,Citation13 In addition, medical educators have been tasked to develop and fit new content into an already heavy curricular load.Citation11,Citation13 Often, medical educators have limited resources, training, or experiential skills in curriculum development.Citation14 Opportunities to address these challenges and integrate HSS into undergraduate, graduate, and continuing medical education is possible through gaming science innovations (GSIs), as outlined in this paper. We hope this serves as a tool to help educators incorporate HSS into medical education in interesting and engaging ways.

Gaming science innovations

Gaming science refers to the use of games as training tools to engage learners through motivational, cognitive reasoning, and metacognition components to advance scientific thinking and skill acquisition.Citation15,Citation16 The historical roots of gaming science lie deep in early civilizations evidenced by war games, chess, and other serious games to promote strategic and critical thinking skills necessary for defense against opposing threats and forces.Citation17 Military and emergency responders have long recognized the importance and benefits of GSIs.Citation16Citation18

Gamification is the application of “game design elements in non-game contexts,”Citation19 such as education.Citation20Citation22 Gamification pertains to the processes of thinking through game elements to solve problems.Citation23 The gamification term, serious game carries definitions, such as “the use of game principles for the purposes of learning, skill acquisition, and training”Citation16 and “the voluntary attempt to overcome necessary obstacles.”Citation16 Existing literature is fundamentally favorable toward gamification uses, benefits, and outcomes.Citation16,Citation22,Citation24Citation31 Educational game innovations are known also as gamified learning interventions,Citation15 gamified training platforms,Citation25 and the aforementioned serious games.Citation16 Video games,Citation18 mobile applications, electronic games,Citation25,Citation28,Citation32 and simulationCitation19,Citation32Citation37 are examples of game designs or learning architectures.Citation16 In health care, where errors can result in dangerous injuries or loss of life, GSIs hold the promise of safe environments for practice, attainment, assessment, and fine-tuning of skills crucial to improve real-life outcomes.Citation17,Citation25 Gaming science principles that strengthen scientific and systems thinkingCitation21 align with the HSS paradigm.

HSS framework

HSS is grounded in Engel’s biopsychosocial model with systems-thinking theoretical roots.Citation9,Citation38,Citation39 The model includes biological, psychological, and sociological life in a hierarchy of systems spanning to the biosphere.Citation39 The HSS conceptual framework focus is on interdependent elements in a biopsychosocial hierarchy. HSS education will incorporate the physiologic factors that influence disease and health outcomes as well as the social environment, economic environment, and the health care system.Citation39 By incorporating HSS into medical education, learners will know how to address the multitude of factors that influence health and well-being.Citation38

The HSS framework identifies 6 core domains and 5 cross-cutting domains, all linked together by the systems thinking domain.Citation9,Citation39

Core domains

  • Health care structures and processes

  • Health care policy, economics, and management

  • Clinical informatics and health information technology

  • Population health

  • Value-based care

  • Health system improvement

Cross-cutting domains

  • Leadership and change agency

  • Teamwork and interprofessional education

  • Evidence-based medicine and practice

  • Professionalism and ethics

  • Scholarship

Linking domain

Core domain: health care structures and processes

The health care structures and processes domain pertains to matters that affect patients, providers, care systems, resources, or processes.Citation9,Citation39 Broad determinants of care quality include patient satisfaction, nature of interactions with staff, personalized care, accessibility to care, timeliness of care, and teamwork.Citation40 Knowledge of this domain is important to promote, care for, and maintain healthy communities. For a generation of health care workers used to video games and social media, training games set in the virtual worlds or virtual reality are cognitively engaging because they allow trainees to construct new knowledge through mediated immersion.Citation41

The Envision Community Health Center (CHC)© is a gamified health care training platformCitation42 to effectively address HSS domains, with a special emphasis on health care structure and processes. Designed to prepare health care teams to provide compassionate, holistic, and evidence-based care to medically underserved populations, medical students complete 26 community-oriented primary care cases related to 10 patient families from diverse ethnicities.Citation43 Gamified elements include dramatic stories set in a virtual world, a scoring system, and virtual preceptors.Citation43 Specific to the first HSS core domain, first- and second-year medical students become oriented with patients, teams, resources, care structures, and processes of Federally Qualified CHCs as they practice care in these settings.Citation43

The process of care can be gamified to provide risk-free practice. Virtual reality simulation is one GSI platform to promote competency development and mastery of processes,Citation33 such as endoscopy,Citation36 prescribing guidelines,Citation29,Citation34 clinical evaluations, and surgical decision making.Citation30,Citation33,Citation37 For example, virtual reality training with endoscopic simulators provides learners opportunities to practice new procedures and advance skills prior to performing procedures on patients.Citation36 Surgical trainees have benefited from reality-based simulation and gamification to gain competency in HSS content areas, such as patient workup and operative skills.Citation30 An iPAD game, PAtient Safety in Surgical EDucation (PASSED), that mimics real-world hospital sentinel events increased medical students’ situational awareness and understanding of patient safety concerns in clinical practice.Citation44

Opportunities exist through simulationCitation34 and Internet-based training modules to advance clinical competenceCitation45 in knowledge acquisition, patient education, and processes, such as prescription guideline adherence to address the opioid crisisCitation34 and other urgent health concerns.Citation29 Simulation stories serve as teaching, learning, and research tools to examine risk evaluation and mitigation strategies for pain and opioid management, in which conflict was minimized when participants were given time to interact with simulated patients.Citation35 Advantages to teaching and learning these skills with GSIs are immediate feedback and increased motivation as players pursue a goal. Games can be designed to challenge learners, so progress to the next level is dependent upon mastering specific competencies.Citation46

Core domain: health care policy, economics, and management

This domain encompasses many factors affecting decisions, actions, and plans put forward to achieve specific health care goals.Citation9,Citation39 Further, this domain includes the components of value, cost, and quality. To understand this domain, students explore policy and health care finance, insurance markets, and forms of reimbursement. Students further evaluate how these factors influence consumer and provider behavior, systems, and health outcomes.Citation9,Citation39

Audience response systems are GSIs used successfully to engage learners in large group lectures for many topics.Citation28 These can be used for curricular integration of the HSS domain of health policy, economics, and management. Scenarios can be presented where learners choose the next step or determine the optimal response to various inputs. Gamified audience response systems are effective tools to promote active learning, real-time feedback, critical thinking, and problem solving of complex content.Citation28

The British Columbia Patient Safety & Quality Council (BCPSQC) created a province-wide campaign using gamification to encourage clinicians to participate in education and adoption of sepsis identification policies and management tools.Citation26 Success of the gamification learning module was demonstrated when providers, residents, and medical students applied new knowledge and evidence-based policies in the real-life screening of nearly 1000 patients in emergency departments for sepsis. Over 150 lives were saved during the 150-day sepsis screening phase of the campaign. British Columbia reported sustained reduction in sepsis-related mortality after the campaign period. The BCPSQC attributed the campaign success to the use of positive intrinsic and extrinsic game elements as an effective management approach to bring about behavioral changes and policy compliance.Citation26

GSIs have also been effective in engaging learners in role-playingCitation16 and acquisition of pharmacological management skillsCitation20,Citation29,Citation47 in alignment with health care policy, economics, and management. Games where students and trainees navigate as avatars through virtual scenarios, such as how to manage patient complaints and drug shortages in health centers and other social settingsCitation20 are innovative techniques to integrate HSS content into education and practice. A study in the UK of gamified simulation cases found statistically significant decreases in unsafe medication errors when medical students followed care policies to manage medical emergencies.Citation29 The role-playing game, Dr. Fill, includes a tool for learners to self-evaluate the ability to accurately recognize, sort, and use quick-response barcodes for medication distribution. The goal is to increase patient safety by reducing medication errors.Citation47 GSIs that promote management skills can also lead to learner awareness of how minimizing errors reduces health costs while improving economics and health outcomes.

Core domain: clinical informatics/health information technology

This HSS domain focuses on informatics sciences, clinical decision-support systems, and electronic health record (EHR) system function and interoperability.Citation9,Citation39 Reporting of various health data is required at national, state, and local levels.Citation48 Measurement of health data is essential to improve patient, public, and population health.Citation49 Thus, it is increasingly important to have methods to educate providers to use rapidly changing technology. In a systematic review of gaming science literature, researchers concluded GSIs promote role- and team-based best practices, build confidence, and reduce errors in EHR users.Citation27 In the Envision CHC©,Citation42 EHR and clinical decision-support systems mimic real-life review and practice with health information technology and informatics.Citation43 To address this domain, GSI designers can incorporate free EHR and data management resources,Citation27,Citation50 such as Blue ButtonCitation51 and PracticeFusion.Citation52 These features provide learners with opportunities to advance understanding and competency in health information and clinical decision support technologies for the documentation, continuity, and delivery of quality care.Citation53

Core domain: population health

The population health domain is inclusive of public health principles for the promotion of equitable health, wellness, and access to quality care for all people.Citation9,Citation39 Gaming science principles that prompt metacognition and behavioral changesCitation26 can advance HSS education related to population health. The potential for GSI use extends to calls for innovative approaches to address patient safety and population health emergencies, such as the current opioid crisisCitation34 and potential epidemic or pandemic outbreaks.Citation54 The POD game,Citation55 Outbreak at the Watersedge©,Citation56 The Great Flu Game and Medical MysteriesCitation57 are interactive educational games that address public health disasters, such as bioterrorism and disease outbreaks.

GSIs that guide learners to explore social determinants of health and population health needs in decision-making have been found to be effective training tools in medical schools,Citation42,Citation43 athletic training programs,Citation58 and nursing programs.Citation50,Citation59 The CDC Foundation offers a premier serious game kit that addresses socioeconomic, behavioral, environmental and other social determinants of health.Citation60 Health and Well-Being for All engages health professions students in collaborative, interactive, case studies. These focus on the social and economic determinants of health. Study topics include navigating the health care system for chronic diseases, such as asthma or obesity in a social environment of poverty, food insecurity, or gang violence. In this GSI, students role play to experience concerns from varying stakeholder perspectives in a community context.Citation60

Population health services tailored to meet specific needs might be identified by engaging patients and communities in GSIs.Citation61 The Wii Fit Plus, for example, tracks activity for improved function of people who have experienced a stroke, multiple sclerosis, or other debilitating health concerns. During this type of gamified care, patients and care teams collaborate in rehabilitation activities to advance HSS content, such as social dynamics for improved compliance and population health outcomes.Citation61

Core domain: value-based care

Value-based care is an important component of the Quadruple Aim for improvement of population health, patient care experiences, provider work life, and cost reduction.Citation9,Citation12,Citation39 New approaches are needed to prepare medical students, residents, and physicians to support value-based care.Citation62 GSIs are a creative method to promote learner awareness of value as it relates to improved performance for patient and provider experiences, costs, and other quality measures.

The gamification platform, Health eRide: Your Journey to Managing Pain, has served as a useful intervention and measurement tool to assess veterans’ readiness to self-manage chronic pain.Citation31 Platforms, such as the medical mobile application, Prognosis: Your Diagnosis©,Citation63,Citation64 and the simulation-based, DecisionSim™Citation64,Citation65 provide risk-free deliberate practice for simple and complex clinical presentations where cost-effective decisions are encouraged and value-based care is promoted.Citation64 The IHI along with the Scottish National Health Service developed and tested a continuous value management innovation in one of the largest hospitals in Scotland.Citation66 The result was a successful value improvement system that included the use of value management boxes and Plan-Do-Study-Act cycles.Citation66 GSI design replication of these action-oriented tools and activities is one method to crystallize team performance measures, reduce waste, and promote value management concepts to learners.

Core domain: health system improvement

Health system improvement pertains to the analyses of root causes for effective change management.Citation9,Citation39 Further, this domain involves processes to identify, analyze, or implement changes in policy, care delivery, or care systems to improve performance.Citation39 As health systems constantly evolve and strive for improvements, GSIs offer opportunities for sustained engagement of all stakeholders in health from patients to provider to administrators.Citation26

The IHI Open School asserts that games are unique tools to learn about improvement.Citation54 A number of health system improvement games, such as Emergency Department Flow, Health Care Scattegories, and The Paper Airplane Factory are available for download on the IHI Open School Web site.Citation54 Game elements and feedback can be customized for stakeholders to identify, analyze, solve, and improve potential health system scenarios like those in Edugame, which take place in various hospital departments.Citation46 A study of the cardiology and nephrology departments in the hospital phase of Edugame demonstrated that the game enhanced student’s motivation and knowledge acquisitionCitation46 for transformative practice and system improvement.

Cross-cutting and linking domains

HSS cross-cutting domains are knowledge and skill areas that overlap with core content domains.Citation9,Citation39 For example, leadership and change agency are necessary to effectively adapt to evolving health policies and systems. The cross-cutting domain of teamwork and interprofessional education is a key area where GSIs can be used. GSIs can foster role- and team-based, interprofessional education opportunities.Citation25,Citation27 Interprofessional education that models how teams behave ethically, learn patient safety protocols, and function efficiently in systems is essentialCitation26 to accelerate HSS teambuilding and skill acquisition for quality improvement.

Systems thinking is a way to view, understand, examine, analyze, and improve the quality of system parts and interactionsCitation67 within and between levels of health systems and the larger social and economic contexts.Citation9 The application of systems thinking brings about a scientific habit of mind.Citation67 In a proposed theory of work gamification, the type and length of applications translate into systems thinking, an informational pathway, to increase effectiveness and motivation and result in improved performance.Citation68 Connections between factors that influence health from micro- to macro-level systems become apparent to learners who navigate through the Envision CHC© cases,Citation42 for example. GSIs designed to link HSS with basic and clinical science are effective methods to accelerate learner awareness, systems thinking, and application of Quadruple Aim goals to enhance patient and provider care experiences, improve population health, and control costs.Citation9,Citation12,Citation38,Citation39

Limitations and implications

Although there are well-documented benefits, GSIs are not without flaws or limitations. All potential human, system, and natural factors cannot be accounted for in gamified learning platforms.Citation17 Human conditions, such as fear, system decisions outside individual control like warfare strategies, and natural disasters are historical examples of unpredictable wargaming deficits,Citation17 which also apply to health care simulation. Improved population health involves countless interdependent variables, such as behaviors, systems, and cultures among other social determinants of health.Citation1,Citation69 Gaming science is rapidly progressing as an andragogical approach to teaching and learning.Citation22,Citation25,Citation27 Research is required to examine facets of this growing phenomenon.

Advocates and critics of GSIs in health profession education propose further research into the relationships between gaming experiences and translation to real-life clinical practice.Citation15,Citation22,Citation25,Citation27,Citation32,Citation45,Citation47,Citation61,Citation70 Research to investigate the perceptions of learners, GSI designers, educators, and administrators is crucial to understand opportunities and challenges to GSI adoption in curricula and training environments.Citation15,Citation22,Citation70,Citation71 RelevanceCitation71 and cost-effectivenessCitation32,Citation33,Citation72 as well as learners’ design element preferences, learning styles,Citation15,Citation16,Citation71 age, and medical predispositions, such as physical disabilities, are GSI research topics.Citation61 Longitudinal studies of HSS integration and its influence on the application of knowledge, attitudes, and skills in medical students, trainees, and practitioners are important to validate GSI effectiveness in medical education.

Conclusion

With more emphasis on integration of HSS education, providers of the future will be better equipped to improve individual and population health outcomes and experiences while reducing costs and improving care team well-being. Achievement of these Quadruple Aim goals is dependent on transformed health profession education, training, and practice. There is much to accomplish as the Accelerating Change in Medical Education Consortium and other stakeholders work to promote the art and science of medicine and improve the nation’s health. Faculty development, resources, and quality improvement are essential to design, deliver, and evaluate curriculum for an evolving basic, clinical, and HSS triad. Although not a panacea for all scholarly endeavors, gaming science offers educators fresh perspectives and opportunities to advance HSS integration in the continuum of medical education, training, and practice. Historical accounts of serious games and current evidence support the potential use of gaming science for safe learning environments to practice, assess, and gain proficiency for the goal of improving real-life outcomes.

Disclosure

The authors report no conflicts of interest in this work.

References

  • LewisNPopulations, population health, and the evolution of population management: making sense of the terminology in U.S. health care todayInstitute for Healthcare Improvement Available from: http://www.ihi.org/communities/blogs/population-health-population-management-terminology-in-us-health-care3192014Accessed December 6, 2017
  • MagnanSSocial determinants of health 101 for health care: five plus five National Academy of Medicine Available from: https://nam.edu/social-determinants-of-health-101-for-health-care-five-plus-five/1092017Accessed December 6, 2017
  • Health systems: aboutWorld Health Organization Available from: http://www.who.int/healthsystems/about/en/Accessed November 13, 2017
  • About us: innovationsInstitute for Healthcare Improvement Available from: http://www.ihi.org/about/pages/innovationscontributions.aspx. Updated 2017Accessed November 13, 2017
  • DzauVJMcClellanMBurkeSVital directions for health and health care: priorities from a National Academy of Medicine initiativeJAMA2017317141461147028324029
  • BaciuASharfsteinJMPopulation health case reports from clinic to communityJAMA2016315242663266427135870
  • EgglestonEMFinkelsteinJAFinding the role of health care in population healthJAMA2014311879779824457610
  • SkochelakSEHawkinsRELawsonLEStarrSRBorkanJMGozaloJDAMA Education Consortium Health Systems Science1st edPhiladelphia, PAElsevier2017
  • GonzaloJDDekhtyarMStarrSRHealth systems science curricula in undergraduate medical education: identifying and defining a potential curricular frameworkAcad Med201792112313127049541
  • American Medical AssociationCreating a Community of InnovationChicago, ILAmerican Medical Association2017
  • GonzaloJDCaverzagieKJHawkinsRLawsonLWolpawDRChangAConcerns and responses for integrating health systems science into medical educationAcad Med Epub20171024
  • BodenheimerTSinskyCFrom triple to quadruple aim: care of the patient requires care of the providerAnn Fam Med201412657357625384822
  • GonzaloJDHaidetPBlattBWolpawDRExploring challenges in implementing a health systems science curriculum: a qualitative analysis of student perceptionsMed Educ20165052353127072441
  • ThomasPAKernDEHughsMTChenBYCurriculum Development: A Six-Step Approach for Medical Education3rd edBaltimore, MDJohn Hopkins University Press2016
  • BuckleyPDoyleEO’MahoneyAIndividualising gamification: investigating how learning styles impact upon gamificationComput Educ20171064355
  • CainJPiascikPAre serious games a good strategy for pharmacy education?Am J Pharm Educ2015794471626089556
  • CaffreyMCJrToward a history-based doctrine for wargamingAerospace Power J20001433356
  • AllenRAmerica’s Army and the military recruitment and management of ‘talent’: an interview with Colonel Casey WardynskiJGVW201462179191
  • DeterdingSDixonDKhaledRNackeLFrom game design elements to gamefulness: defining “gamification”Proceedings for the 15th International Academic MindTrek Conference: Envisioning Future Media EnvironmentsNew YorkACM2011915
  • YapKY-LYapKZYapJYGA gamification framework for training of patient and medications management skillsJILR2015114758
  • MorrisBJCrokerSZimmermanCGillDRomigCGaming science: the “Gamification” of scientific thinkingFront Psychol2013460711623382719
  • Sanchez-MenaAMarti-ParrenoJDrivers and barriers to adopting gamification: teachers’ perspectivesEJEL2017155434443
  • ZichermannGCunninghamCGamification by Design: Implementing Game Mechanics in Web and Mobile AppsSebastopol, CAO’Reilly Media, Inc2011
  • CudneyEAMurraySLSpragueCMEngaging healthcare users through gamification in knowledge sharing of continuous improvement in healthcareProcedia Manuf2015334163423
  • McCoyLLewisJHDaltonDGamification and multimedia for medical education: a landscape reviewJ Am Osteopath Assoc20161161223426745561
  • McKeownSKrauseCShergillMSiuASweetDGamification as a strategy to engage and motivate clinicians to improve careHealth Manage Forum20162926773
  • GibbsDHewittBMcLeodAThe gamification of electronic health records: a systematic literature reviewEPHIIM2016112
  • PettitRKMcCoyLKinneyMSchwartzFNStudent perceptions of gamified response system interactions in large group lectures and via lecture capture technologyBMC Med Educ2015159225997953
  • WoodfieldGO’SullivanMHaddingtonNStantonAUsing simulation for prescribing: an evaluationClin Teach201411242824405915
  • LinDTParkJLiebertCALauJNValidity evidence for surgical improvement of clinical knowledge ops: a novel gaming platform to assess surgical decision makingAm J Surg2015209798525454955
  • JohnsonSLevesqueDBroderickLHealth eRide: outcomes of a pilot program leveraging game principles of gamification and SMS messaging to help veterans self-manage chronic painJMIR20162129
  • McGaghieWCIssenbergSBPetrusaERScaleseRJRevisiting ‘a critical review of simulation-based medical education research: 2003–2009’Med Educ20165098699127628715
  • Price KerfootBKissaneNThe use of gamification to boost residents’ engagement in simulation trainingJAMA Surg20141491111208120925229631
  • BoyleKLTeaching effective opioid prescribing through a simulation curriculumAnn Emerg Med2015664sS157
  • HeirichMMackeySNewmarkJUse of simulation as a teaching and research tool for opioid management: implications for patient skillsJ Pain2017184S49S50
  • KochADEkkelenkampVEHaringsmaJSchoonEJde ManRAKuipersEJSimulated colonoscopy training leads to improved performance during patient-based assessmentGastrointest Endosc201581363063625475901
  • BambakidisNCSelmanWRSloanAESurgical rehearsal platform: potential uses in microsurgeryNeurosurgery2013731S122S126
  • BorkanJMWolpawTMStagg ElliottVThe future of health systems scienceSkochelakSEHawkinsRELawsonLEStarrSRBorkanJMGozaloJDAMA Education Consortium Health Systems Science1st edPhiladelphia, PAElsevier2017208216
  • GonzaloJDStarrSRBorkanJMWhat is health systems science? Building an integrated visionSkochelakSEHawkinsRELawsonLEStarrSRBorkanJMGozaloJDAMA Education Consortium Health Systems Science1st edPhiladelphia, PAElsevier20171023
  • CowingMDavion-RamayaCMRamayaKSzmerekovskyJHealth care delivery performance: service, outcomes, and resource stewardshipPerm J20081347278
  • DedeCPlanning for neomillennial learning stylesEducause Q2005281712
  • Envision Community Health Center© Available from: https://vchc.atsu.edu/Accessed December 7, 2017
  • McCoyLLewisJHBennettTAllgoodJBayCSchwartzFNFostering service orientation in medical students through a virtual community health centerJ Family Med Community Health20163219
  • KowAWCAngBLSChongCSTanWBMenonKRInnovation patient safety curriculum using iPAD game (PASSED) improved patient safety concepts in undergraduate medical studentsWorld J Surg2016402571258027417109
  • KimHHeverlingHCordeiroMVasquezVStolbachAInternet training resulted in improved trainee performance in a simulated opioid-poisoned patient as measured by checklistJ Med Toxicol20161228929427037564
  • PesareERoselliTCorrieroNRossanoVGame-based learning and gamification to promote engagement and motivation in medical contextsSmart Learn Environ201635
  • BukowskiMKuhnMZhaoXBettermannRJonasSGamification of clinical routine: the Dr. Fill approachStud Health Technol Inform201622526226627332203
  • Health Center Data & ReportingHealth Resources & Services Administration: HRSA Health Center Program Available from: https://bphc.hrsa.gov/datareporting/. Updated November 2017Accessed March 14, 2018
  • Population & Public HealthThe Office of the National Coordinator for Health Information Technology Available from: https://www.healthit.gov/playbook/population-public-health/. Updated February 28, 2018Accessed March 14, 2018
  • CostelloECorcoranMBarnettJSInformation and communication technology to facilitate learning for students in the health professions: current uses, gaps, and future directionsOLJ2014184
  • Blue Button. HealthIT.gov Available from: https://www.healthit.gov/patients-families/your-health-dataAccessed December 2, 2017
  • PracticeFusionPractice Fusion Resource Center Available from: https://www.practicefusion.com/practice-resource-center/Accessed December 2, 2017
  • KimJClinical decision support tools still underused by clinicians2014TechTarget Available from: http://searchhealthit.techtarget.com/feature/Clinical-decision-support-tools-still-underused-by-cliniciansAccessed March 18, 2018
  • Games and ExercisesInstitute for Health Care Improvement Open School Available from: http://www.ihi.org/education/IHIOpenSchool/resources/Pages/Tools/default.aspxAccessed March 12, 2018
  • The POD game Chicago Department of Public Health and University of Illinois at Chicago School of Public Health Center for the Advancement of Distance Education Available from: http://thepodgame.com/Accessed March 16, 2018
  • Outbreak at the Watersedge©University of Minnesota Available from: http://www.mclph.umn.edu/watersedge/Accessed March 16, 2018
  • Games. Cambridge Infectious Diseases, An Interdisciplinary Research Centre at the University of Cambridge Available from: https://www.infectiousdisease.cam.ac.uk/schoolzone/games/games-images/Great%20Flu%20Game.jpg/viewAccessed March 16, 2018
  • WalkerSWeidnerTArmstrongKJStandardized patient encounters and individual case-based simulations improve students’ confidence and promote reflection: a preliminary studyJ Athl Ther Train2015102130137
  • ReimeMHJohnsgaardTKvamFISimulated settings; powerful arenas for learning patient safety practices and facilitating transference to clinical practice. A mixed method studyNurse Educ Pract201621758227769018
  • Health and Well-Being for AllCDC Foundation Available from: https://www.cdcfoundation.org/health-in-a-boxAccessed March 16, 2018
  • HammediWLeclerqTVanRielACRThe use of gamification mechanics to increase employee and user engagement in participative healthcare services: a study of two casesJOSM2017284640661
  • HavyerRDNorbySMLeep HunderfundANScience of health care delivery milestones for undergraduate medical educationBMC Med Educ2017171451628056975
  • Prognosis: Your Diagnosis©Medical Joyworks Available from: http://www.prognosisapp.com/Accessed December 6, 2017
  • McCoyLPettitRKLewisJHDeveloping technology-enhanced active learning for medical education: challenges, solutions, and future directionsJ Am Osteopath Assoc2015115420221125830576
  • DecisionSim Technology™Kynectiv Available from: https://app.decisionsimulation.com/shell/Login.aspxAccessed December 6, 2017
  • 10 IHI Innovations to improve health and health careCambridge, MAInstitute for Healthcare Improvement2017 Available from: www.ihi.orgAccessed March 16, 2018
  • PetersDHThe application of systems thinking in health: why uses systems thinking?Health Res Policy Syst2014125125160707
  • CardadorMTNorthcraftGBWhickerJA theory of work gamification: something old, something new, something borrowed, something cool?HRMR201727353365
  • HoodCMGennusoKPSwainGRCatlinBBCounty health rankings: relationships between determinant factors and health outcomesAm J Prev Med201650212913526526164
  • RojasDKapralosBDubrowskiAThe role of game elements in online learning within health professions educationStud Health Technol Inform201622032933427046600
  • Galbis-CordovaAMarti-ParrenoJCurras-PerezRHigher education students’ attitude towards the use of gamification for competencies developmentJe-LKS2017131129146
  • ZendejasBWangATBrydgesRHamstraSJCookDACost: the missing outcome in simulation-based medical education research: a systematic reviewSurgery2013153216017622884087