An iterative adaptive developmental evaluation framework: case study of the TREE semester learning community

References

• Aguilar, O. M., and M. E. Krasny. 2011. “Using the Communities of Practice Framework to Examine an after-School Environmental Education Program for Hispanic Youth.” Environmental Education Research 17 (2): 217–233. https://doi.org/10.1080/13504622.2010.531248
   Web of Science ®Google Scholar
• Anderson, K. C., M. J. Stern, R. B. Powell, A. A. Dayer, and T. G. Archibald. 2022. “A Culturally Responsive Evaluation Framework and Its Application in Environmental Education.” Evaluation and Program Planning 92: 102073. https://doi.org/10.1016/j.evalprogplan.2022.102073
   PubMed Web of Science ®Google Scholar
• Ardoin, N. M., K. Biedenweg, and K. O’Connor. 2015. “Evaluation in Residential Environmental Education: An Applied Literature Review of Intermediary Outcomes.” Applied Environmental Education & Communication 14 (1): 43–56. https://doi.org/10.1080/1533015X.2015.1013225
   Google Scholar
• Arias, J. M., and H. Drossman. 2022. “Power & Equity in Undergraduate Environmental Education.” Clearing Magazine, Winter 2021, 24–27.
   Google Scholar
• Awuah, R., S. Agyepong, and P. Kwarase. 2020. “Leadership Education through Extracurricular Civic Engagement.” Partnerships: A Journal of Service-Learning and Civic Engagement 11 (1): 257–274.
   Google Scholar
• Barbera, J., and J. R. VandenPlas. 2011. “All Assessment Materials Are Not Created Equal: The Myths about Instrument Development, Validity, and Reliability.” In Investigating Classroom Myths through Research on Teaching and Learning, edited by D. M. Bunce, 177–193. Washington, DC: ACS Publications.
   Google Scholar
• Bird, Á., F. Fahy, and K. Reilly. 2022. “Making Evaluation Work for the Practitioner Evaluator: Experience from the Field of Environmental Education.” Environmental Education Research 28 (5): 715–734. https://doi.org/10.1080/13504622.2022.2036324
   Web of Science ®Google Scholar
• Boeve-de Pauw, J. 2014. “Moving Environmental Education Forward through Evaluation.” Studies in Educational Evaluation 41: 1–3. doi.org/jgn8 https://doi.org/10.1016/j.stueduc.2014.04.002
   Web of Science ®Google Scholar
• Brousselle, A., and J. McDavid. 2021. “Evaluation for Planetary Health.” Evaluation 27 (2): 168–183. https://doi.org/10.1177/1356389020952462
   Web of Science ®Google Scholar
• Buck, J., and S. Villines. 2017. We the People: Consenting to a Deeper Democracy; a Guide to Sociocratic Principles and Methods. 2nd Ed. Washington, DC: Sociocracy.info.
   Google Scholar
• Carifio, J., and R. J. Perla. 2007. “Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and Their Antidotes.” Journal of Social Sciences 3 (3): 106–116. https://doi.org/10.3844/jssp.2007.106.116
   Google Scholar
• Carpenter, S. K., A. E. Witherby, and S. K. Tauber. 2020. “On Students’(Mis) Judgments of Learning and Teaching Effectiveness.” Journal of Applied Research in Memory and Cognition 9 (2): 137–151. https://doi.org/10.1016/j.jarmac.2019.12.009
   Web of Science ®Google Scholar
• Cawthorn, M., L. Leege, and E. Congdon. 2011. “Improving Learning Outcomes in Large Environmental Science Classrooms through Short-Term Service-Learning Projects.” Journal of Environmental Studies and Sciences 1 (1): 75–87. https://doi.org/10.1007/s13412-011-0001-8
   Google Scholar
• Chase, F. S. (1962). “Some Effects of Current Curriculum Projects on Educational Policy and Practice.” The School Review 70 (1): 132–147.
   Google Scholar
• Crohn, K., and M. Birnbaum. 2010. “Environmental Education Evaluation: Time to Reflect, Time for Change.” Evaluation and Program Planning 33 (2): 155–158. https://doi.org/10.1016/j.evalprogplan.2009.07.004
   PubMed Web of Science ®Google Scholar
• Dawson, T. L. 2020. “Rethinking Educational Assessment in Light of a Strong Theory of Development.” In Handbook of Integrative Developmental Psychology, edited by M. E. Biddell, 423–450. New York: Routledge. doi.org/bskm5w
   Google Scholar
• Drossman, H., J. Benedict, E. McGrath-Spangler, L. Van Roekel, and K. Wells. 2011. “Assessment of a Constructivist-Motivated Mentoring Program to Enhance the Teaching Skills of Atmospheric Science Graduate Students.” Journal of College Science Teaching 41 (2): 72–81.
   Google Scholar
• Drossman, H., S. O. Gaia, and T. Dawson. 2015. “Development of an Integral-Motivated Lectical® Ecological Stewardship Assessment.” Paper presented at the Integral Theory Conference, July 17, 2015, Santa Rosa, CA.
   Google Scholar
• Eoyang, G. H., and R. J. Holladay. 2013. Adaptive Action: Leveraging Uncertainty in Your Organization. Stanford, CA: Stanford University Press.
   Google Scholar
• Eoyang, G., and J. Oakden. 2016. “Adaptive Evaluation: A Synergy between Complexity Theory and Evaluation Practice.” Emergence: Complexity and Organization 18 (3/4): 1–14. doi:10.emerg/10.17357.e5389f5715a734817dfbeaf25ab335e5
   Google Scholar
• Ernst, J. A., M. C. Monroe, and B. Simmons. 2009. Evaluating Your Environmental Education Programs: A Workbook for Practitioners. Washington, DC: North American Association for Environmental Education.
   Google Scholar
• Galiazzi, M. C., P. D. Salomão de Freitas, C. Aguiar de Lima, C. da Silva Cousin, M. Langoni de Souza, and R. Launikas Cupelli. 2018. “Narratives of Learning Communities in Environmental Education.” Environmental Education Research 24 (10): 1501–1513. https://doi.org/10.1080/13504622.2018.1545152
   Web of Science ®Google Scholar
• Gough, A. 2013. “The Emergence of Environmental Education Research: A “History” of the Field.” In International Handbook of Research on Environmental Education, edited by R. B. Stevenson, M. Brody, J. Dillon, and A. E. Wals, 13–22. New York: Routledge.
   Google Scholar
• Harness, H., and H. Drossman. 2011. “The Environmental Education through Filmmaking Project.” Environmental Education Research 17 (6): 829–849. https://doi.org/10.1080/13504622.2011.618626
   Web of Science ®Google Scholar
• Huber, E., and M. Harvey. 2013. “Time to Participate: Lessons from the Literature for Learning and Teaching Project Evaluation in Higher Education.” Studies in Educational Evaluation 39 (4): 240–249. https://doi.org/10.1016/j.stueduc.2013.10.004
   Web of Science ®Google Scholar
• Jenks, B., P. W. Vaughan, and P. J. Butler. 2010. “The Evolution of Rare Pride: Using Evaluation to Drive Adaptive Management in a Biodiversity Conservation Organization.” Evaluation and Program Planning 33 (2): 186–190. https://doi.org/10.1016/j.evalprogplan.2009.07.010
   PubMed Web of Science ®Google Scholar
• Kools, M., and L. Stoll. 2016. “What Makes a School a Learning Organisation?” OECD Education Working Papers, no. 137. Washington, DC: OECD Publishing.
   Google Scholar
• Korthagen, F. A. 2010. “How Teacher Education Can Make a Difference.” Journal of Education for Teaching 36 (4): 407–423. https://doi.org/10.1080/02607476.2010.513854
   Web of Science ®Google Scholar
• Krasny, M. E. 2020. Advancing Environmental Education Practice. Ithaca, NY: Cornell University Press.
   Google Scholar
• Lakens, D. 2013. “Calculating and Reporting Effect Sizes to Facilitate Cumulative Science: A Practical Primer for t-Tests and ANOVAs.” Frontiers in Psychology 4 (Article 863). https://doi.org/10.3389/fpsyg.2013.00863
   PubMedGoogle Scholar
• Lave, J. 2018. “The Practice of Learning.” In Contemporary Theories of Learning: Learning Theorists… in Their Own Words, edited by K. Illeris, 200–208. New York: Routledge.
   Google Scholar
• Lu, Y.-L L., and C.-W. Wu. 2018. “An Integrated Evaluation Model of Teaching and Learning.” Journal of University Teaching and Learning Practice 15 (3): 116–133. https://ro.uow.edu.au/jutlp/vol15/iss3/8. https://doi.org/10.53761/1.15.3.8
   Web of Science ®Google Scholar
• Mayhew, K. C., and A. C. Edwards. 1966. The Dewey School: The Laboratory School of the University of Chicago, 1896-1903. New Brunswick, NJ: Transaction Publishers.
   Google Scholar
• Meadows, D. H. 2008. Thinking in Systems: A Primer. White River Junction, VT: Chelsea Green Publishing.
   Google Scholar
• Mitchell, C., and L. Sackney. 2011. Profound Improvement: Building Capacity for a Learning Community. New York: Routledge.
   Google Scholar
• Monroe, M. C. 2010. “Challenges for Environmental Education Evaluation.” Evaluation and Program Planning 33 (2): 194–196. https://doi.org/10.1016/j.evalprogplan.2009.07.012
   PubMed Web of Science ®Google Scholar
• Monroe, L., and D. M. Horm. 2012. “Using a Logic Model to Evaluate Undergraduate Instruction in a Laboratory Preschool.” Early Education & Development 23 (2): 227–241. https://doi.org/10.1080/10409289.2012.619135
   Web of Science ®Google Scholar
• NAAEE. 2019. Professional Development of Environmental Educators: Guidelines for Excellence. 5th ed. Washington, DC: North American Association for Environmental Education.
   Google Scholar
• Nicholas-Figueroa, L., R. Hare, M. van Muelken, L. Duffy, and C. Middlecamp. 2017. “Iḷisaġvik Tribal College’s Summer Climate Program: Teaching STEM Concepts to North Slope Alaska High School and Middle-School Students.” Journal of Environmental Studies and Sciences 7 (3): 425–434. https://doi.org/10.1007/s13412-016-0413-6
   Google Scholar
• Norman, G. 2010. “Likert Scales, Levels of Measurement and the “Laws” of Statistics.” Advances in Health Sciences Education 15 (5): 625–632. https://doi.org/10.1007/s10459-010-9222-y
   PubMed Web of Science ®Google Scholar
• Owen, R. L., and J. A. Buck. 2020. “Creating the Conditions for Reflective Team Practices: Examining Sociocracy as a Self-Organizing Governance Model That Promotes Transformative Learning.” Reflective Practice 21 (6): 786–802. https://doi.org/10.1080/14623943.2020.1821630
   Web of Science ®Google Scholar
• Pallant. 2020. SPSS Survival Manual. London: Open University Press.
   Google Scholar
• Patton, M. Q. 1994. “Developmental Evaluation.” Evaluation Practice, 15 (3): 311–319. https://doi.org/10.1177/109821409401500312
   Google Scholar
• Patton, M. Q. 2011. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: Guilford Press.
   Google Scholar
• Patton, M. Q. 2016. “What is Essential in Developmental Evaluation? On Integrity, Fidelity, Adultery, Abstinence, Impotence, Long-Term Commitment, Integrity, and Sensitivity in Implementing Evaluation Models.” American Journal of Evaluation 37 (2): 250–265. https://doi.org/10.1177/1098214015626295
   Web of Science ®Google Scholar
• Patton, M. Q. 2020. Blue Marble Evaluation: Premises and Principles. New York: Guilford Press.
   Google Scholar
• Patton, M. Q., K. McKegg, and N. Wehipeihana. 2016. Developmental Evaluation Exemplars: Principles in Practice. New York: Guilford Press.
   Google Scholar
• Powell, R. B., M. J. Stern, and N. Ardoin. 2006. “A Sustainable Evaluation Framework and Its Application.” Applied Environmental Education & Communication 5 (4): 231–241. https://doi.org/10.1080/15330150601059290
   Google Scholar
• Preston, L. 2012. “An Outdoor & Environmental Education Community of Practice: Self Stylisation or Normalisation?” Journal of Outdoor and Environmental Education 16 (1): 28–38. https://doi.org/10.1007/BF03400936
   Google Scholar
• Rau, T. J., and J. Koch-Gonzalez. 2018. Many Voices, One Song. Amherst, MA: Sociocracy for All.
   Google Scholar
• Rowe, A. 2019. “Sustainability-Ready Evaluation: A Call to Action.” New Directions for Evaluation 2019 (162): 29–48. https://doi.org/10.1002/ev.20365
   Google Scholar
• Saldaña, J. 2016. The Coding Manual for Qualitative Researchers. Thousand Oaks, CA: Sage.
   Google Scholar
• Scriven, M. 1991. Evaluation Thesaurus. 4th ed. Thousand Oaks, CA: Sage.
   Google Scholar
• Senge, P. M. 2006. The Fifth Discipline: The Art and Practice of the Learning Organization. 2nd ed. New York: Currency.
   Google Scholar
• Senge, P. M., N. Cambron-McCabe, T. Lucas, B. Smith, and J. Dutton. 2012. Schools That Learn (Updated and Revised): A Fifth Discipline Fieldbook for Educators, Parents, and Everyone Who Cares about Education. New York: Currency.
   Google Scholar
• Seymour, E., D. Wiese, A. Hunter, and S. M. Daffinrud. 2000. “Creating a Better Mousetrap: On-Line Student Assessment of Their Learning Gains.” Paper Presented at the National Meeting of the American Chemical Society.
   Google Scholar
• Stapleton, S. R. 2020. “Toward Critical Environmental Education: A Standpoint Analysis of Race in the American Environmental Context.” Environmental Education Research 26 (2): 155–170. https://doi.org/10.1080/13504622.2019.1648768
   Web of Science ®Google Scholar
• Steyn, C., C. Davies, and A. Sambo. 2019. “Eliciting Student Feedback for Course Development: The Application of a Qualitative Course Evaluation Tool among Business Research Students.” Assessment & Evaluation in Higher Education 44 (1): 11–24. https://doi.org/10.1080/02602938.2018.1466266
   Web of Science ®Google Scholar
• Sullivan, L. 2019. TREE Semester Program Evaluation: Integrating Learning Theories and Application in Practice. Colorado Springs, CO: Colorado College.
   Google Scholar
• Tan, C. 2020. “The Learning School through a Daoist Lens.” Oxford Review of Education 46 (3): 393–407. https://doi.org/10.1080/03054985.2020.1714571
   Web of Science ®Google Scholar
• Tanner, L. 1997. Dewey’s Laboratory School. New York: Teachers College Press.
   Google Scholar
• Thomas, V. G., and B. A. Parsons. 2017. “Culturally Responsive Evaluation Meets Systems-Oriented Evaluation.” American Journal of Evaluation 38 (1): 7–28. https://doi.org/10.1177/1098214016644069
   Web of Science ®Google Scholar
• Tight, M. 2021. “Existing Research on Learning Gain in Higher Education.” In Learning Gain in Higher Education (International Perspectives on Higher Education Research), edited by C. Hughes and M. Tight, 1–16. Vol. 14. Leeds: Emerald Publishing Limited.
   Google Scholar
• Tunks, T. 1978. “The Use of Omega Squared in Interpreting Statistical Significance.” Bulletin of the Council for Research in Music Education 28–34.
   Web of Science ®Google Scholar
• Waapalaneexkweew. 2018. “Looking Backward but Moving Forward: Honoring the Sacred and Asserting the Sovereign in Indigenous Evaluation.” American Journal of Evaluation 39 (4): 543–568.
   Web of Science ®Google Scholar
• Wenger, E. 1999. Communities of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge University Press.
   Google Scholar
• Westhorp, G. 2012. “Using Complexity-Consistent Theory for Evaluating Complex Systems.” Evaluation 18 (4): 405–420. https://doi.org/10.1177/1356389012460963
   Web of Science ®Google Scholar
• Westhorp, G., K. Stevens, and P. J. Rogers. 2016. “Using Realist Action Research for Service Redesign.” Evaluation 22 (3): 361–379. https://doi.org/10.1177/1356389016656514
   Web of Science ®Google Scholar
• Williams, R. 2018. “Does the Colorado College TREE Semester Model Ecofeminist Pedagogy?: A Qualitative Case Study.” Unpublished Undergraduate Thesis, Colorado College.
   Google Scholar
• Zint, M. T., P. F. Dowd, and B. A. Covitt. 2011. “Enhancing Environmental Educators’ Evaluation Competencies: Insights from an Examination of the Effectiveness of the My Environmental Education Evaluation Resource Assistant (MEERA) Website.” Environmental Education Research, 17 (4): 471–497. https://doi.org/10.1080/13504622.2011.565117
   Web of Science ®Google Scholar