The Strength of Evidence Pyramid: One Approach for Characterizing the Strength of Evidence of Geoscience Education Research (GER) Community Claims

ABSTRACT

During the past two decades, the Geoscience Education Research (GER) community has been increasingly recognized as an evidence-based research subdiscipline in the geoscience and in the larger discipline-based education research (DBER) field. Most recently, the GER community has begun to address the current state of the field and discuss the best course forward so that it can have the greatest collective impact on advancing teaching and learning in the geosciences. The community has formally recognized that practice should be evidence based and that the strengths and limitations of community-level research claims should be transparent. As such, this commentary article describes a conceptual model—the Strength of Evidence Pyramid—as a pathway to organize the strength of evidence in the GER community of generalizable claims generated by both geo–Scholarship of Teaching and Learning and geo-DBER efforts. Its design is informed by a rubric and the outcomes of a DBER synthesis, as well as by parallels we see in the concept of evidence-based medicine in the health sciences. The proposed GER Strength of Evidence Pyramid uses five levels to categorize GER-community claims: (1) practitioner wisdom/expert opinion; original qualitative and quantitative studies, including (2) case studies and (3) cohort studies; and analyzed published literature in the form of (4) meta-analyses and (5) systematic reviews. The goal of the Pyramid is to assist geoscience-education researchers and geoscience educators to visualize, organize their thinking, and evaluate the quality of the evidence of GER-community claims. The potential applications and limitations of the model for use in the GER community are described.

Key words: :

• strength of evidence
• Strength of Evidence Pyramid
• generalizability of findings

Acknowledgments

This work has benefitted from thoughtful discussions with Heather Macdonald, Kim Kastens, Tony Feig, Laura Lukes, Eric Riggs, and participants in the 2015 GER workshop at the Earth Educators' Rendezvous. We also thank the Editor and reviewers for their constrictive comments, which improved the description and discussion of the proposed GER Strength of Evidence Pyramid. The initial idea for this conceptual model stemmed from planning activities for the 2015 GER workshop, which was funded by the National Science Foundation under grant DUE-1513519 (principal investigator, Heather Macdonald), Shaping the Future of Geoscience Education Research: Synthesizing Results and Articulating Future Directions.

Notes

³ At present, the term DBER has only been applied to education research in STEM disciplines (Henderson et al., 2017), but the possibility of DBER in non-STEM disciplines also exists (Singer et al 2012).

⁴ NAGT GER Division Web site: http://nagt.org/nagt/divisions/geoed/index.html.

⁵ http://nagt.org/nagt/geoedresearch/index.html.

⁶ See http://www.phcris.org.au/guides/about_research.php [PHCRIS, 2017].

⁷ See http://www.dartmouth.edu/∼biomed/resources.htmld/guides/ebm_psych_resources.html.

⁸ See http://libguides.gwumc.edu/ebm/studytypes.

⁹ 2015 GER Workshop information: http://serc.carleton.edu/earth_rendezvous/2015/morning_workshops/w3/index.html

¹⁰ https://serc.carleton.edu/NAGTWorkshops/coursedesign/index.html.

¹¹ http://nagt.org/nagt/geoedresearch/toolbox/index.html.

¹² This idea was introduced by Kim Kastens at the 2017 Earth Educators Rendezvous Geoscience Education Research and Practice Forum: https://serc.carleton.edu/earth_rendezvous/2017/program/ger/index.html.

¹³ https://serc.carleton.edu/index.html.

¹⁴ https://serc.carleton.edu/teachearth/index.html.

¹⁵ https://serc.carleton.edu/NAGTWorkshops/index.html.

¹⁶ JGE literature review guidelines, http://nagtjge.org/userimages/ContentEditor/1447705357878/JGE_Lit_Rev_Guidelines_Nov13_2015.pdf

¹⁷ GSA Repository https://www.geosociety.org/datarepository/.

¹⁸ World Data Service for Paleoclimatology for ice core data https://www.ncdc.noaa.gov/data-access/paleoclimatology-data/datasets/ice-core.

¹⁹ Databrary https://nyu.databrary.org/.

²⁰ Levels of evidence and grades of recommendations, https://hsl.lib.umn.edu/biomed/help/levels-evidence-andgrades-recommendations

²¹ University of Pittsburgh Health Sciences Library System, https://hsls.pitt.edu/resources/ebm.

²² Walden University Library evidence-based practice research http://academicguides.waldenu.edu/healthevidence/evidencepyramid#s-lg-box-8700027,

²³ https://serc.carleton.edu/index.html.