Aligning multiple choice assessments with active learning instruction: More accurate and equitable ways to measure student learning

References

• Aguillon, S. M., Siegmund, G. F., Petipas, R. H., Drake, A. G., Cotner, S., & Ballen, C. J. (2020). Gender differences in student participation in an active-learning classroom. CBE—Life Sciences Education, 19(2), ar12. https://doi.org/10.1187/cbe.19-03-0048
   PubMed Web of Science ®Google Scholar
• Aird, H., & Teasdale, R. (2017). Determining the factors that affect the effective incorporation of active-learning techniques into a large introductory geology class. [Poster presentation]. GSA Annual Meeting, Seattle, Washington, USA.
   Google Scholar
• Allison, J., & Eversole, R. (2008). A new direction for regional university campuses: Catalyzing innovation in place. Innovation: The European Journal of Social Science Research, 21(2), 95–109.
   Web of Science ®Google Scholar
• Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.
   Google Scholar
• Ballen, C. J. (2020). Enhancing diversity in college science with active learning. In Active learning in college science (pp. 873–887). Springer.
   Google Scholar
• Ballen, C. J., Wieman, C., Salehi, S., Searle, J. B., & Zamudio, K. R. (2017). Enhancing diversity in undergraduate science: Self-efficacy drives performance gains with active learning. CBE—Life Sciences Education, 16(4), ar56. https://doi.org/10.1187/cbe.16-12-0344
   PubMed Web of Science ®Google Scholar
• Bandura, A. (1986). The explanatory and predictive scope of self-efficacy theory. Journal of Social and Clinical Psychology, 4(3), 359–373. https://doi.org/10.1521/jscp.1986.4.3.359
   Web of Science ®Google Scholar
• Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3), 347–364. https://doi.org/10.1007/BF00138871
   Web of Science ®Google Scholar
• Biggs, J., & Tang, C. (2011). Train-the-trainers: Implementing outcomes-based teaching and learning in Malaysian higher education. Malaysian Journal of Learning and Instruction, 8, 1–19.
   Google Scholar
• Bloom, B. S. (1956). Taxonomy of educational objectives (Vol. 1). Cognitive domain (vol. 20, pp. 1). McKay.
   Google Scholar
• Boyle, A. P. (2007). Using alignment and reflection to improve student learning. Elements, 3(2), 113–117. https://doi.org/10.2113/gselements.3.2.113
   Web of Science ®Google Scholar
• Brownell, S. E., & Tanner, K. D. (2012). Barriers to faculty pedagogical change: Lack of training, time, incentives, and… tensions with professional identity? CBE—Life Sciences Education, 11(4), 339–346. https://doi.org/10.1187/cbe.12-09-0163
   PubMed Web of Science ®Google Scholar
• Carnegie Classification. (2019). The Carnegie Classification of Institutions of Higher Education. http://carnegieclassifications.ie.edu/
   Google Scholar
• Carrell, S. E., Page, M. E., & West, J. E. (2010). Sex and science: How professor gender perpetuates the gender gap. The Quarterly Journal of Economics, 125(3), 1101–1144.
   Web of Science ®Google Scholar
• Chávez, A. F., & Longerbeam, S. D. (2016). Teaching across cultural strengths: A guide to balancing integrated and individuated cultural frameworks in college teaching. Stylus Publishing, LLC.
   Google Scholar
• Chemers, M. M., Hu, L. T., & Garcia, B. F. (2001). Academic self-efficacy and first year college student performance and adjustment. Journal of Educational Psychology, 93(1), 55–64. [Database] https://doi.org/10.1037/0022-0663.93.1.55
   Web of Science ®Google Scholar
• Chi, M. T. (2009). Active-constructive-interactive: a conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105. https://doi.org/10.1111/j.1756-8765.2008.01005.x
   PubMed Web of Science ®Google Scholar
• Chi, M. T., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823
   Web of Science ®Google Scholar
• Choi, H. H., Van Merriënboer, J. J., & Paas, F. (2014). Effects of the physical environment on cognitive load and learning: Towards a new model of cognitive load. Educational Psychology Review, 26(2), 225–244.
   Web of Science ®Google Scholar
• Cohen, S. A. (1987). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16(8), 16–20. https://doi.org/10.3102/0013189X016008016
   Google Scholar
• Cotner, S., & Ballen, C. J. (2017). Can mixed assessment methods make biology classes more equitable? PloS One, 12(12), e0189610. https://doi.org/10.1371/journal.pone.0189610
   PubMed Web of Science ®Google Scholar
• Crowe, A., Dirks, C., & Wenderoth, M. P. (2008). Biology in bloom: Implementing Bloom’s taxonomy to enhance student learning in biology. CBE Life Sciences Education, 7(4), 368–381.
   PubMed Web of Science ®Google Scholar
• CSU, Chico Institutional Research. (2021). Enrollment at a glance. Retrieved September 2021, from, https://wildcats-bi-ext.csuchico.edu/t/InstitutionalResearch/views/Enrollment_0/EnrollmentataGlance?iframeSizedToWindow=true&%3Aembed=y&%3AshowAppBanner=false&%3Adisplay_count=no&%3AshowVizHome=no
   Google Scholar
• Darling-Hammond, L. (2014). Closing the achievement gap: A systemic view. In Closing the achievement gap from an international perspective (pp. 7–20). Springer.
   Google Scholar
• Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrollment physics class. Science, 332(6031), 862–864. [Database] https://doi.org/10.1126/science.1201783
   PubMed Web of Science ®Google Scholar
• Di Giusto, F., Müller Werder, C., Reichmuth, A., Adams-Hausheer, D., & Christian, J. (2019). Multiple-choice questions: teaching guide for higher and professional education. Retrieved 2021 from https://digitalcollection.zhaw.ch/bitstream/11475/19339/1/Multiple-choice%20questions_en.pdf
   Google Scholar
• DiBenedetto, M. K., & Bembenutty, H. (2013). Within the pipeline: Self-regulated learning, self-efficacy, and socialization among college students in science courses. Learning and Individual Differences, 23, 218–224. https://doi.org/10.1016/j.lindif.2012.09.015
   Web of Science ®Google Scholar
• Dolin, J., Black, P., Harlen, W., & Tiberghien, A. (2018). Exploring relations between formative and summative assessment. In Transforming assessment (pp. 53–80). Springer.
   Google Scholar
• Dutt, K. (2020). Race and racism in the geosciences. Nature Geoscience, 13(1), 2–3. https://doi.org/10.1038/s41561-019-0519-z
   Web of Science ®Google Scholar
• Dyar, M. D. (2012). Gender and geoscience specialization as a function of object and spatial visualization skills. Geological Society of America Special Papers, 486, 79–83.
   Google Scholar
• Ebert-May, D., Derting, T. L., Henkel, T. P., Middlemis Maher, J., Momsen, J. L., Arnold, B., & Passmore, H. A. (2015). Breaking the cycle: Future faculty begin teaching with learner-centered strategies after professional development. CBE—Life Sciences Education, 14(2), ar22. https://doi.org/10.1187/cbe.14-12-0222
   PubMed Web of Science ®Google Scholar
• Eddy, S. L., & Brownell, S. E. (2016). Beneath the numbers: A review of gender disparities in undergraduate education across science, technology, engineering, and math disciplines. Physical Review Physics Education Research, 12(2), 020106. https://doi.org/10.1103/PhysRevPhysEducRes.12.020106
   Web of Science ®Google Scholar
• Eddy, S. L., Brownell, S. E., & Wenderoth, M. P. (2014). Gender gaps in achievement and participation in multiple introductory biology classrooms. CBE Life Sciences Education, 13(3), 478–492.
   PubMed Web of Science ®Google Scholar
• Eddy, S. L., & Hogan, K. A. (2014). Getting under the hood: How and for whom does increasing course structure work? CBE Life Sciences Education, 13(3), 453–468.
   PubMed Web of Science ®Google Scholar
• Fink, L. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses. John Wiley & Sons.
   Google Scholar
• Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111
   PubMed Web of Science ®Google Scholar
• Freeman, S., Haak, D., & Wenderoth, M. P. (2011). Increased course structure improves performance in introductory biology. CBE Life Sciences Education, 10(2), 175–186.
   PubMed Web of Science ®Google Scholar
• Friedlander, L. J., Reid, G. J., Shupak, N., & Cribbie, R. (2007). Social support, self-esteem, and stress as predictors of adjustment to university among first-year undergraduates. Journal of College Student Development, 48(3), 259–274. [Database] https://doi.org/10.1353/csd.2007.0024
   Web of Science ®Google Scholar
• Fry, C. (Ed.). (2014). Achieving systemic change: A sourcebook for advancing and funding and funding undergraduate STEM education. Association of American Colleges and Universities.
   Google Scholar
• Gold, A. U., Pendergast, P. M., Ormand, C. J., Budd, D. A., Stempien, J. A., Mueller, K. J., & Kravitz, K. A. (2018). Spatial skills in undergraduate students—Influence of gender, motivation, academic training, and childhood play. Geosphere, 14(2), 668–683. https://doi.org/10.1130/GES01494.1
   Web of Science ®Google Scholar
• Goldsmith, D. W. (2011). A case-based curriculum for introductory geology. Journal of Geoscience Education, 59(3), 119–125. https://doi.org/10.5408/1.3604824
   Google Scholar
• Gonzalez, L. (2010). Minority participation in university programs, AGI Geoscience Currents, No. 30 http://www.agiweb.org/workforce/Currents/Currents-030-URM_GeoDegrees.pdf
   Google Scholar
• Gray, K., Steer, D., McConnell, D., & Owens, K. (2010). Using a student-manipulated model to enhance student learning in a large lecture class. Journal of College Science Teaching, 40(1), 86.
   Google Scholar
• Greene, T. G., Marti, C. N., & McClenney, K. (2008). The effort—outcome gap: Differences for African American and hispanic community college students in student engagement and academic achievement. The Journal of Higher Education, 79(5), 513–539. https://doi.org/10.1080/00221546.2008.11772115
   Web of Science ®Google Scholar
• Gross, D., Pietri, E. S., Anderson, G., Moyano-Camihort, K., & Graham, M. J. (2015). Increased preclass preparation underlies student outcome improvement in the flipped classroom. CBE Life Sciences Education, 14(4), ar36.
   PubMed Web of Science ®Google Scholar
• Gutiérrez, R. (2008). Research Commentary: A gap-gazing fetish in mathematics education? Problematizing research on the achievement gap. Journal for Research in Mathematics Education, 39(4), 357–364. https://doi.org/10.5951/jresematheduc.39.4.0357
   Web of Science ®Google Scholar
• Haak, D. C., HilleRisLambers, J., Pitre, E., & Freeman, S. (2011). Increased structure and active learning reduce the achievement gap in introductory biology. Science (New York, NY), 332(6034), 1213–1216. https://doi.org/10.1126/science.1204820
   Google Scholar
• Halpern, D. F., & Hakel, M. D. (2003). Applying the science of learning to the university and beyond: Teaching for long-term retention and transfer. Change: The Magazine of Higher Learning, 35(4), 36–41. https://doi.org/10.1080/00091380309604109
   Google Scholar
• Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., Chang, A., DeHaan, R., Gentile, J., Lauffer, S., Stewart, J., Tilghman, S. M., & Wood, W. B. (2004). Education. Scientific teaching. Science (New York, NY), 304(5670), 521–522. https://doi.org/10.1126/science.1096022
   Google Scholar
• Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I., & Hyde, J. S. (2014). Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106(2), 375–389.
   PubMed Web of Science ®Google Scholar
• Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Priniski, S. J., & Hyde, J. S. (2016). Closing achievement gaps with a utility-value intervention: Disentangling race and social class. Journal of Personality and Social Psychology, 111(5), 745–765.
   PubMed Web of Science ®Google Scholar
• Harris, M. T. (2002). Developing geoscience student-learning centered courses. Journal of Geoscience Education, 50(5), 515–523. https://doi.org/10.5408/1089-9995-50.5.515
   Google Scholar
• Harris, R. B., Grunspan, D. Z., Pelch, M. A., Fernandes, G., Ramirez, G., & Freeman, S. (2019). Can test anxiety interventions alleviate a gender gap in an undergraduate STEM course? CBE Life Sciences Education, 18(3), ar35. https://doi.org/10.1187/cbe.18-05-0083
   PubMed Web of Science ®Google Scholar
• Henderson, C., & Dancy, M. H. (2007). Barriers to the use of research-based instructional strategies: The influence of both individual and situational characteristics. Physical Review Special Topics-Physics Education Research, 3(2), 020102. [Database][Mismatch
   Web of Science ®Google Scholar
• Hobbs, H. T., Singer-Freeman, K. E., & Robinson, C. (2021). Considering the effects of assignment choices on equity gaps. Research & Practice in Assessment, 16(1), 49–62.
   Google Scholar
• Holt, E. A., Young, C., Keetch, J., Larsen, S., & Mollner, B. (2015). The greatest learning return on your pedagogical investment: Alignment, assessment or in-class instruction? PloS One, 10(9), e0137446. https://doi.org/10.1371/journal.pone.0137446
   PubMed Web of Science ®Google Scholar
• Hubbard, J. K., Potts, M. A., & Couch, B. A. (2017). How question types reveal student thinking: An experimental comparison of multiple-true-false and free-response formats. CBE—Life Sciences Education, 16(2), ar26. https://doi.org/10.1187/cbe.16-12-0339
   Web of Science ®Google Scholar
• Hurtado, S., Eagan, K., Pryor, J. H., Whang, H., & Tran, S. (2012). Undergraduate teaching faculty: The 2010–2011 HERI faculty survey. Higher Education Research Institute: University of California.
   Google Scholar
• Hurtado, S., Newman, C. B., Tran, M. C., & Chang, M. J. (2010). Improving the rate of success for underrepresented racial minorities in STEM fields: Insights from a national project. New Directions for Institutional Research, 2010(148), 5–15. https://doi.org/10.1002/ir.357
   Google Scholar
• Hyytinen, H., Nissinen, K., Ursin, J., Toom, A., & Lindblom-Ylänne, S. (2015). Problematising the equivalence of the test results of performance-based critical thinking tests for undergraduate students. Studies in Educational Evaluation, 44, 1–8. https://doi.org/10.1016/j.stueduc.2014.11.001
   Web of Science ®Google Scholar
• InTeGrate. (2017). Make content relevant to students’ lives. Retrieved November, 2017, from https://serc.carleton.edu/integrate/teaching_materials/using/online/relevant.html
   Google Scholar
• James, N. M., Kreager, B. Z., & LaDue, N. D. (2022). Predict-observe-explain activities preserve introductory geology students’ self-efficacy. Journal of Geoscience Education, 70(2), 238–249.
   Google Scholar
• Jensen, J. L., McDaniel, M. A., Woodard, S. M., & Kummer, T. A. (2014). Teaching to the test… or testing to teach: Exams requiring higher order thinking skills encourage greater conceptual understanding. Educational Psychology Review, 26(2), 307–329. https://doi.org/10.1007/s10648-013-9248-9
   Web of Science ®Google Scholar
• Jewett, S., Sutphin, K., Gierasch, T., Hamilton, P., Lilly, K., Miller, K., Newlin, D., Pires, R., Sherer, M., & LaCourse, W. R. (2018). Awareness, analysis, and action: Curricular alignment for student success in general chemistry. Journal of Chemical Education, 95(2), 242–247. https://doi.org/10.1021/acs.jchemed.7b00564
   Web of Science ®Google Scholar
• Jordt, H., Eddy, S. L., Brazil, R., Lau, I., Mann, C., Brownell, S. E., King, K., & Freeman, S. (2017). Values affirmation intervention reduces achievement gap between underrepresented minority and white students in introductory biology classes. CBE—Life Sciences Education, 16(3), ar41. https://doi.org/10.1187/cbe.16-12-0351
   Web of Science ®Google Scholar
• Kacprzyk, J., Parsons, M., Maguire, P. B., & Stewart, G. S. (2019). Examining gender effects in different types of undergraduate science assessment. Irish Educational Studies, 38(4), 467–480. https://doi.org/10.1080/03323315.2019.1645721
   Web of Science ®Google Scholar
• Kahl, J. D. (2008). Reflections on a large-lecture, introductory meteorology course: Goals, assessment, and opportunities for improvement. Bulletin of the American Meteorological Society, 89(7), 1029–1034. https://doi.org/10.1175/2008BAMS2473.1
   Web of Science ®Google Scholar
• Kastens, K. A., Manduca, C. A., Cervato, C., Frodeman, R., Goodwin, C., Liben, L. S., Mogk, D. W., Spangler, T. C., Stillings, N. A., & Titus, S. (2009). How geoscientists think and learn. Eos, Transactions American Geophysical Union, 90(31), 265–266. https://doi.org/10.1029/2009EO310001
   Google Scholar
• Kirschner, F., Paas, F., Kirschner, P. A., & Janssen, J. (2011). Differential effects of problem-solving demands on individual and collaborative learning outcomes. Learning and Instruction, 21(4), 587–599. https://doi.org/10.1016/j.learninstruc.2011.01.001
   Web of Science ®Google Scholar
• Kober, N. (2015). Reaching students: What research says about effective instruction in undergraduate science and engineering. National Academies Press.
   Google Scholar
• Ku, K. Y. (2009). Assessing students’ critical thinking performance: Urging for measurements using multi-response format. Thinking Skills and Creativity, 4(1), 70–76. https://doi.org/10.1016/j.tsc.2009.02.001
   Web of Science ®Google Scholar
• Lauer, S., Momsen, J., Offerdahl, E., Kryjevskaia, M., Christensen, W., & Montplaisir, L. (2013). Stereotyped: Investigating gender in introductory science courses. CBE Life Sciences Education, 12(1), 30–38.
   PubMed Web of Science ®Google Scholar
• Lewis, P. J. (2016). Brain friendly teaching-reducing learner’s cognitive load. Academic Radiology, 23(7), 877–880. https://doi.org/10.1016/j.acra.2016.01.018
   PubMed Web of Science ®Google Scholar
• Leyva, L. A., Quea, R., Weber, K., Battey, D., & López, D. (2021). Detailing racialized and gendered mechanisms of undergraduate precalculus and calculus classroom instruction. Cognition and Instruction, 39(1), 1–34. https://doi.org/10.1080/07370008.2020.1849218
   Web of Science ®Google Scholar
• Liu, J. C., John, K. S., & Courtier, A. M. B. (2017). Development and validation of an assessment instrument for course experience in a general education integrated science course. Journal of Geoscience Education, 65(4), 435–454. https://doi.org/10.5408/16-204.1
   Google Scholar
• Lombardi, D., Shipley, T. F., Bailey, J. M., Bretones, P. S., Prather, E. E., Ballen, C. J., Knight, J. K., Smith, M. K., Stowe, R. L., Cooper, M. M., Prince, M., Atit, K., Uttal, D. H., LaDue, N. D., McNeal, P. M., Ryker, K., St. John, K., van der Hoeven Kraft, K. J., & Docktor, J. L., & Astronomy Team, Biology Team, Chemistry Team, Engineering Team, Geography Team, Geoscience Team, and Physics Team. (2021). The curious construct of active learning. Psychological Science in the Public Interest, 22(1), 8–43. https://doi.org/10.1177/1529100620973974
   Web of Science ®Google Scholar
• Lorenzo, M., Crouch, C. H., & Mazur, E. (2006). Reducing the gender gap in the physics classroom. American Journal of Physics, 74(2), 118–122. https://doi.org/10.1119/1.2162549
   Web of Science ®Google Scholar
• Lukes, L. A., & McConnell, D. A. (2014). What motivates introductory geology students to study for an exam? Journal of Geoscience Education, 62(4), 725–735. https://doi.org/10.5408/13-110.1
   Google Scholar
• MacPhee, D., Farro, S., & Canetto, S. S. (2013). Academic self‐efficacy and performance of underrepresented STEM majors: Gender, ethnic, and social class patterns. Analyses of Social Issues and Public Policy, 13(1), 347–369. https://doi.org/10.1111/asap.12033
   Web of Science ®Google Scholar
• Matz, R. L., Koester, B. P., Fiorini, S., Grom, G., Shepard, L., Stangor, C. G., Weiner, B., & McKay, T. A. (2017). Patterns of gendered performance differences in large introductory courses at five research universities. AERA Open, 3(4), 233285841774375–233285841774312. https://doi.org/10.1177/2332858417743754
   Web of Science ®Google Scholar
• McConnell, D. A., Chapman, L., Czajka, C. D., Jones, J. P., Ryker, K. D., & Wiggen, J. (2017). Instructional utility and learning efficacy of common active learning strategies. Journal of Geoscience Education, 65(4), 604–625. https://doi.org/10.5408/17-249.1
   Google Scholar
• McConnell, D. A., Steer, D. N., & Owens, K. D. (2003). Assessment and active learning strategies for introductory geology courses. Journal of Geoscience Education, 51(2), 205–216. https://doi.org/10.5408/1089-9995-51.2.205
   Google Scholar
• McConnell, D. A., Stempien, J. A., Perkins, D., van der, H., Kraft, K. J., Vislova, T., Wirth, K. R., Budd, D. A., Bykerk-Kauffman, A., Gilbert, L. A., & Matheney, R. K. (2010). The little engine that could—less prior knowledge but high self-efficacy is equivalent to greater prior knowledge and low self-efficacy. Geological Society of America Abstracts with Programs, 42(5), 191.
   Google Scholar
• McGee, E., & Bentley, L. (2017). The equity ethic: Black and Latinx college students reengineering their STEM careers toward justice. American Journal of Education, 124(1), 1–36. https://doi.org/10.1086/693954
   Web of Science ®Google Scholar
• McKenzie, K., & Schweitzer, R. (2001). Who succeeds at university? Factors predicting academic performance in first year Australian university students. Higher Education Research & Development, 20(1), 21–33. https://doi.org/10.1080/07924360120043621
   Google Scholar
• Michael, J. (2006). Where’s the evidence that active learning works? Advances in Physiology Education, 30(4), 159–167. https://doi.org/10.1152/advan.00053.2006
   PubMed Web of Science ®Google Scholar
• Mihram, D., & Mihram, G. A. (2011). The AAAS 2011 Annual: Science without borders. Library Hi Tech News, 28(7), 7–13. https://www.aaas.org/resources/aaas-annual-report-2011 https://doi.org/10.1108/07419051111184034
   Google Scholar
• Milner, H. R.IV, (2012). Beyond a test score: Explaining opportunity gaps in educational practice. Journal of Black Studies, 43(6), 693–718. https://doi.org/10.1177/0021934712442539
   Web of Science ®Google Scholar
• Miyake, A., Kost-Smith, L. E., Finkelstein, N. D., Pollock, S. J., Cohen, G. L., & Ito, T. A. (2010). Reducing the gender achievement gap in college science: A classroom study of values affirmation. Science (New York, N.Y.), 330(6008), 1234–1237.
   PubMed Web of Science ®Google Scholar
• Momsen, J. L., Long, T. M., Wyse, S. A., & Ebert-May, D. (2010). Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills. CBE Life Sciences Education, 9(4), 435–440.
   PubMed Web of Science ®Google Scholar
• Momsen, J., Offerdahl, E., Kryjevskaia, M., Montplaisir, L., Anderson, E., & Grosz, N. (2013). Using assessments to investigate and compare the nature of learning in undergraduate science courses. CBE Life Sciences Education, 12(2), 239–249.
   PubMed Web of Science ®Google Scholar
• Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41), 16474–16479. https://doi.org/10.1073/pnas.1211286109
   PubMed Web of Science ®Google Scholar
• National Center for Education Statistics (NCES). (2019). Fast facts: SAT scores. https://nces.ed.gov/fastfacts/display.asp?id=171
   Google Scholar
• Nehm, R. H., & Schonfeld, I. S. (2008). Measuring knowledge of natural selection: A comparison of the CINS, an open response instrument, and an oral interview. Journal of Research in Science Teaching, 45(10), 1131–1160. https://doi.org/10.1002/tea.20251
   Web of Science ®Google Scholar
• Nicol, D. (2007). Laying a foundation for lifelong learning: Case studies of e‐assessment in large 1st‐year classes. British Journal of Educational Technology, 38(4), 668–678. https://doi.org/10.1111/j.1467-8535.2006.00657.x
   Web of Science ®Google Scholar
• Núñez, A. M., Rivera, J., & Hallmark, T. (2020). Applying an intersectionality lens to expand equity in the geosciences. Journal of Geoscience Education, 68(2), 97–114. https://doi.org/10.1080/10899995.2019.1675131
   Google Scholar
• O’Connell, S., & Holmes, M. A. (2011). Obstacles to the recruitment of minorities into the geosciences: A call to action. GSA Today, 21(6), 52–54. https://doi.org/10.1130/G105GW.1
   Google Scholar
• Oleson, A., & Hora, M. T. (2014). Teaching the way they were taught?, Revisiting the sources of teaching knowledge and the role of prior experience in shaping faculty teaching practices. Higher Education, 68(1), 29–45. https://doi.org/10.1007/s10734-013-9678-9
   Web of Science ®Google Scholar
• Olson, S., & Riordan, D. G. (2012). Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Report to the President. Executive Office of the President.
   Google Scholar
• Paas, F., & Ayres, P. (2014). Cognitive load theory: A broader view on the role of memory in learning and education. Educational Psychology Review, 26(2), 191–195. https://doi.org/10.1007/s10648-014-9263-5
   Web of Science ®Google Scholar
• Pai, M. R. S. M., Sanji, N., Pai, P. G., & Kotian, S. (2010). Comparative assessment in Pharmacology multiple choice questions versus essay with focus on gender differences. Journal of Clinical and Diagnostic Research, 4(3), 2515–2520.
   Google Scholar
• Palmer, E. J., & Devitt, P. G. (2007). Assessment of higher order cognitive skills in undergraduate education: Modified essay or multiple choice questions? BMC Medical Education, 7(1), 1–7. https://doi.org/10.1186/1472-6920-7-49
   PubMedGoogle Scholar
• Panchal, P., Prasad, B., & Kumari, S. (2018). Multiple choice questions- role in assessment of competency of knowledge in anatomy. International Journal of Anatomy and Research, 6(2.1), 5156–5162. https://doi.org/10.16965/ijar.2018.143
   Google Scholar
• Pico, T. (2019). Body metaphors in field geology: Implications for gender- or sex- based harassment. American Geophysical Union. Fall Meeting 2019, abstract #ED33F-1038
   Google Scholar
• Pico, T. (2021). Linking past to present in a postcolonial field science: How scientific training and practice in US geology perpetuates marginalization. (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20210301-130908950
   Google Scholar
• Pink, A., & Newton, P. M. (2020). Decorative animations impair recall and are a source of extraneous cognitive load. Advances in Physiology Education, 44(3), 376–382.
   Web of Science ®Google Scholar
• Pintrich, P. R., & Zusho, A. (2007). Student motivation and self-regulated learning in the college classroom. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 731–810). Springer.
   Google Scholar
• Potter, W., Webb, D., Paul, C., West, E., Bowen, M., Weiss, B., Coleman, L., & De Leone, C. (2014). Sixteen years of collaborative learning through active sense-making in physics (CLASP) at UC Davis. American Journal of Physics, 82(2), 153–163. https://doi.org/10.1119/1.4857435
   Web of Science ®Google Scholar
• Preszler, R. W. (2009). Replacing lecture with peer-led workshops improves student learning. CBE Life Sciences Education, 8(3), 182–192. https://doi.org/10.1187/cbe.09-01-0002
   PubMed Web of Science ®Google Scholar
• Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223–231. https://doi.org/10.1002/j.2168-9830.2004.tb00809.x
   Web of Science ®Google Scholar
• Rawlusyk, P. E. (2018). Assessment in higher education and student learning. Journal of Instructional Pedagogies, 21, 1–34.
   Google Scholar
• Reynolds, H. L., & Kearns, K. D. (2017). A planning tool for incorporating backward design, active learning, and authentic assessment in the college classroom. College Teaching, 65(1), 17–27. https://doi.org/10.1080/87567555.2016.1222575
   Google Scholar
• Rosales, J., & Walker, T. (2018). The racist beginnings of standardized testing. National Education Association, 73(2), 88–96.
   Google Scholar
• Russell, J. E., Van Horne, S., Ward, A. S., Bettis, I. I. I., Sipola, E. A., Colombo, M., Rocheford, M., & K, M. (2016). Large lecture transformation: Adopting evidence-based practices to increase student engagement and performance in an introductory science course. Journal of Geoscience Education, 64(1), 37–51. https://doi.org/10.5408/15-084.1
   Google Scholar
• Scully, D. (2017). Constructing multiple-choice items to measure higher-order thinking. Practical Assessment, Research, and Evaluation, 22(1), 4.
   Google Scholar
• Semken, S. (2005). Sense of place and place-based introductory geoscience teaching for American Indian and Alaska Native undergraduates. Journal of Geoscience Education, 53(2), 149–157. https://doi.org/10.5408/1089-9995-53.2.149
   Google Scholar
• Shadle, S. E., Marker, A., & Earl, B. (2017). Faculty drivers and barriers: Laying the groundwork for undergraduate STEM education reform in academic departments. International Journal of STEM Education, 4(1), 1–13. https://doi.org/10.1186/s40594-017-0062-7
   Google Scholar
• Shortlidge, E. E., Rain-Griffith, L., Shelby, C., Shusterman, G. P., & Barbera, J. (2019). Despite similar perceptions and attitudes, postbaccalaureate students outperform in introductory biology and chemistry courses. CBE Life Sciences Education, 18(1), ar3.
   Web of Science ®Google Scholar
• Simkin, M. G., & Kuechler, W. L. (2005). Multiple‐choice tests and student understanding: What is the connection? Decision Sciences. Journal of Innovative Education, 3(1), 73–98.
   Google Scholar
• Singer, S. R., Nielsen, N. R., & Schweingruber, H. A. (2012). Discipline-based education research. The National Academies.
   Google Scholar
• Stanger-Hall, K. F. (2012). Multiple-choice exams: An obstacle for higher-level thinking in introductory science classes. CBE Life Sciences Education, 11(3), 294–306. https://doi.org/10.1187/cbe.11-11-0100
   PubMed Web of Science ®Google Scholar
• Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction, 12(3), 185–233. https://doi.org/10.1207/s1532690xci1203_1
   Web of Science ®Google Scholar
• Teasdale, R., Selkin, P., & Goodell, L. (2018). Evaluation of student learning, self-efficacy, and perception of the value of geologic monitoring from Living on the Edge, an InTeGrate curriculum module. Journal of Geoscience Education, 66(3), 186–204. https://doi.org/10.1080/10899995.2018.1481354
   Google Scholar
• Teasdale, R., van der Hoeven Kraft, K., & Poland, M. P. (2015). Using near-real-time monitoring data from Pu’u’ O’o vent at Kīlauea Volcano for training and educational purposes. Journal of Applied Volcanology, 4(1), 1–16. https://doi.org/10.1186/s13617-015-0026-x
   Google Scholar
• Teasdale, R., Viskupic, K., Bartley, J. K., McConnell, D., Manduca, C., Bruckner, M., Farthing, D., & Iverson, E. (2017). A multidimensional assessment of reformed teaching practice in geoscience classrooms. Geosphere, 13(2), 608–627. https://doi.org/10.1130/GES01479.1
   Web of Science ®Google Scholar
• Teasdale, R., Ryker, K., Viskupic, K., Czajka, Czajka, & Manduca, C. (2020). Transforming education with community-developed teaching materials: evidence from direct observations of STEM college classrooms. International Journal of STEM Education, 7(1), 1–22.
   Web of Science ®Google Scholar
• Tewksbury, B. J., Manduca, C. A., Mogk, D. W., Macdonald, R. H., & Bickford, M. E. (2013). Geoscience education for the Anthropocene. Geological Society of America Special Papers, 501, 189–201.
   Google Scholar
• Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., Chambwe, N., Cintrón, D. L., Cooper, J. D., Dunster, G., Grummer, J. A., Hennessey, K., Hsiao, J., Iranon, N., Jones, L., Jordt, H., Keller, M., Lacey, M. E., Littlefield, C. E., … Freeman, S. (2020). Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences of the United States of America, 117(12), 6476–6483. https://doi.org/10.1073/pnas.1916903117
   PubMed Web of Science ®Google Scholar
• Trujillo, G., & Tanner, K. D. (2014). Considering the role of affect in learning: Monitoring students’ self-efficacy, sense of belonging, and science identity. CBE Life Sciences Education, 13(1), 6–15.
   PubMed Web of Science ®Google Scholar
• van der Hoeven Kraft, K. J. (2017). Developing student interest: An overview of the research and implications for geoscience education research and teaching practice. Journal of Geoscience Education, 65(4), 594–603. https://doi.org/10.5408/16-215.1
   Google Scholar
• Walczyk, J. J., Ramsey, L. L., & Zha, P. (2007). Obstacles to instructional innovation according to college science and mathematics faculty. Journal of Research in Science Teaching, 44(1), 85–106. https://doi.org/10.1002/tea.20119
   Web of Science ®Google Scholar
• Walton, G. M., & Cohen, G. L. (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science (New York, NY), 331(6023), 1447–1451.
   Google Scholar
• Washington, A. N. (2020). When twice as good isn’t enough: The case for cultural competence in computing [Paper presentation]. Proceedings of the 51st ACM Technical Symposium on Computer Science Education. (pp. 213–219).
   Google Scholar
• Weaver, A. J., & Raptis, H. (2001). Gender differences in introductory atmospheric and oceanic science exams: Multiple choice versus constructed response questions. Journal of Science Education and Technology, 10(2), 115–126. https://doi.org/10.1023/A:1009412929239
   Google Scholar
• Webber, K. L. (2012). The use of learner-centered assessment in US colleges and universities. Research in Higher Education, 53(2), 201–228. https://doi.org/10.1007/s11162-011-9245-0
   Web of Science ®Google Scholar
• Weir, L. K., Barker, M. K., McDonnell, L. M., Schimpf, N. G., Rodela, T. M., & Schulte, P. M. (2019). Small changes, big gains: A curriculum-wide study of teaching practices and student learning in undergraduate biology. Plos One, 14(8), e0220900–16. https://doi.org/10.1371/journal.pone.0220900
   PubMed Web of Science ®Google Scholar
• Wenger, E. (2000). Key to knowledge strategy. Knowledge and communities (pp. 3–21). Butterworth-Heinemann.
   Google Scholar
• Wiggins, G., & McTighe, J. (2001). What is backward design? In Understanding by design (1st ed., pp. 7–19). Merrill Prentice Hall.
   Google Scholar
• Wright, C. D., Eddy, S. L., Wenderoth, M. P., Abshire, E., Blankenbiller, M., & Brownell, S. E. (2016). Cognitive difficulty and format of exams predicts gender and socioeconomic gaps in exam performance of students in introductory biology courses. CBE—Life Sciences Education, 15(2), ar23. https://doi.org/10.1187/cbe.15-12-0246
   Web of Science ®Google Scholar
• Wright, V., Kotowski, A. (2021). Mirroring minoritized students’ cultures in Geoscience courses. https://eartharxiv.org/repository/view/2092/
   Google Scholar
• Yuretich, R. F., Khan, S. A., Leckie, R. M., & Clement, J. J. (2001). Active-learning methods to improve student performance and scientific interest in a large introductory oceanography course. Journal of Geoscience Education, 49(2), 111–119. https://doi.org/10.5408/1089-9995-49.2.111
   Google Scholar
• Zusho, A., Pintrich, P. R., & Coppola, B. (2003). Skill and will: The role of motivation and cognition in the learning of college chemistry. International Journal of Science Education, 25(9), 1081–1094. https://doi.org/10.1080/0950069032000052207
   Web of Science ®Google Scholar
• Zwick, R. (2019). Assessment in American higher education: The role of admissions tests. The ANNALS of the American Academy of Political and Social Science, 683(1), 130–148. https://doi.org/10.1177/0002716219843469
   Web of Science ®Google Scholar