Measures, Metrics, and Indicators for Evaluating Technology-Based Interventions

References

• Alkin, M. C. (2004). Evaluation roots: Tracing theorists’ views and influence. SAGE.
   Google Scholar
• Alkin, M. C. (Ed.). (1978). Explorations in evaluation theory development. (CSE Report 123) University of California, Los Angeles, Center for the Study of Evaluation.
   Google Scholar
• American Educational Research Association (AERA), American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Psychological Association.
   Google Scholar
• Baker, E. L. (1974). Formative evaluation of instruction. In W. J. Popham (Ed.), Evaluation in education (pp. 531–585). McCutchan Publishing.
   Google Scholar
• Baker, E. L. (1998). Formative assessment for learning: New goals, new models. In P. Black & A. Michel (Eds.), Learning from pupil assessment: International comparisons (CSE Monograph Series in Evaluation., No. 12, pp. 39–48). University of California, Los Angeles, Center for the Study of Evaluation.
   Google Scholar
• Baker, E. L. (2007). Model-based assessments to support learning and accountability: The evolution of CRESST’s research on multiple-purpose measures. Educational Assessment (Special Issue), 12(3&4), 179–194. https://doi.org/10.1080/10627190701578222
   Google Scholar
• Baker, E. L., & Choi, K. (2018). Force modernization: Training Assessment Framework (Report to funder). University of California.
   Google Scholar
• Baker, E. L., & Choi, K. (2020). Report on generalized process for Training Assessment Framework for selected Navy ratings (Report to funder). University of California.
   Google Scholar
• Baker, E. L., & O’Neil, H. F. (2021, June 2–3). Assessment to support learning: Disrupting a secular faith using principles in motion [Paper presentation]. Edmund W. Gordon Centennial Conference, Teachers College, Columbia University.
   Google Scholar
• Baker, E. L., Cai, L., Choi, K., & Madni, A. (2015, June). Functional validity: Extending the utility of state assessments [Paper presentation]). 2015 National Conference on Student Assessment (NCSA), San Diego, CA, United States.
   Google Scholar
• Baker, E. L., Choi, K., Lee, J. J., Iseli, M. R., Koenig, A. D., & Michiuye, J. K. (2021). Models and tools documentation (Deliverable Item No. 00007 to funder) University of California.
   Google Scholar
• Baker, E. L., Chung, G. K. W. K., & Cai, L. (2016). Assessment gaze, refraction, and blur: The course of achievement testing in the past 100 years. Review of Research in Education, 40(1), 94–142. https://doi.org/10.3102/0091732X16679806
   Google Scholar
• Baker, E. L., Harrison, M. C., & Ervin, B. (2022). Assessment principles for games and innovative technologies. In H. F. O’Neil, E. L. Baker, R. S. Perez, & S. E. Watson (Eds.), Theoretical issues of using simulations and games in educational assessment (pp. 3–37). Routledge/Taylor & Francis.
   Google Scholar
• Baker, E., & Cai, L. (2014, August). CRESST analysis of the quality of a state assessment. [Technical Slide Report]. UCLA/CRESST.
   Google Scholar
• Berman, A. I., Haertel, E. H., & Pellegrino, J. W. (2020). Comparability of large-scale educational assessments: Issues and recommendations. National Academy of Education.
   Google Scholar
• Bewley, W. L., & O’Neil, H. F. (2013). Evaluation of medical simulations. Military medicine, 178(10 Suppl), 64–75. https://doi.org/10.7205/MILMED-D-13-00255
   PubMed Web of Science ®Google Scholar
• Bloom, B. S. (1969). Some theoretical issues relating to educational evaluation. In R. W. Tyler (Ed.), Educational evaluation: New roles, new means (Vol. 68, pp. 26–50). University of Chicago Press. https://doi.org/10.1177/016146816907001003
   Google Scholar
• Choi, K., Parks, C. B., Feng, T., Redman, E. J. K. H., Chung, G. K. W. K. (2021). Molly of Denali analytics validation study report—final (Deliverable to PBS KIDS). UCLA/CRESST. https://ca.pbslearningmedia.org/resource/molly-of-denali-analytics-validation-study-report/rtl-2015-2020-initiative-document/
   Google Scholar
• Chung, G. K. W. K., Delacruz, G. C., Dionne, G. B., & Bewley, W. L. (2006). Linking assessment and instruction using ontologies (CSE Report 693). University of California, Los Angeles, CRESST.
   Google Scholar
• Chung, G. K. W. K., Redman, E. H., Delacruz, G. C., Madni, A., & Lo, G. (2015). Feature analysis ratings—Final (Deliverable to PBS KIDS). UCLA/CRESST.
   Google Scholar
• Cronbach, L. J., & Suppes, P. (1969). Research for tomorrow’s schools: Disciplined inquiry for education. Macmillan.
   Google Scholar
• Ebel, R. L. (1965). Measuring educational achievement. Prentice-Hall.
   Google Scholar
• Elliott, S. (2017). Computers and the future of skill demand. Centre for Educational Research and Innovation, OECD Publishing. https://doi.org/10.1787/9789264284395-en
   Google Scholar
• Glaser, R. (1963). Instructional technology and the measurement of learning outcomes. American Psychologist, 18(8), 519–521. https://doi.org/10.1037/h0049294
   Web of Science ®Google Scholar
• Hively, W., Patterson, H. L., & Page, S. H. (1968). A “universe-defined” system of arithmetic achievement tests. Journal of Educational Measurement, 5(4), 275–290. http://www.jstor.org/stable/1433778
   Web of Science ®Google Scholar
• ISO/IEC/IEEE. (2011). Systems and software engineering—Architecture description. Retrieved from https://www.iso.org/standard/50508.html
   Google Scholar
• Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick’s four levels of training evaluation. Association for Talent Development.
   Google Scholar
• Koenig, A. D., Lee, J. J., Baker, E. L., Michiuye, J. K., & O’Neil, H. F. (2022). Final report (Milestone #8 to funder). University of California.
   Google Scholar
• Lewis, M. (2004). Moneyball: The art of winning an unfair game. W. W. Norton & Company.
   Google Scholar
• Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991). Complex performance-based assessment: Expectations and validation criteria. Educational Researcher, 20(8), 5–21. https://doi.org/10.3102/0013189X020008015
   Google Scholar
• MacLennan, A. (2019, January 8). A complete beginner’s guide to baseball stats: Batting statistics, and what they mean. SB Nation Bless You Boys. https://www.blessyouboys.com/2019/1/8/18171919/baseball-stats-for-beginners-batting-average-on-base-percentage-explained
   Google Scholar
• Markle, S. M. (1978). Designs for instructional designers. Stipes Publishing Company.
   Google Scholar
• Mayer, R. E. (2011). Applying the science of learning. Pearson.
   Google Scholar
• Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (1999). Evidence centered assessments design. Educational Testing Service.
   Google Scholar
• Mousavi, H., Kerr, D., & Iseli, M. R. (2013). Unsupervised ontology generation from unstructured text. (CRESST Report 827). University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
   Google Scholar
• O’Neil, H. F., Baker, E. L., Perez, R. S., & Watson, S. E. (Eds.) (2021). Using cognitive and affective metrics in educational simulations and games: Applications in school and workplace contexts. Routledge/Taylor & Francis.
   Google Scholar
• O’Neil, H. F., Baker, E. L., Perez, R. S., & Watson, S. E. (Eds.) (2022). Theoretical issues of using simulations and games in educational assessment: Applications in school and workplace contexts. Routledge/Taylor & Francis.
   Google Scholar
• O’Neil, H. F., Perez, R. S., & Baker, E. L. (Eds.) (2014). Teaching and measuring cognitive readiness. Springer.
   Google Scholar
• OECD. (2018). Equity in education: Breaking down barriers to social mobility. PISA, OECD Publishing. https://doi.org/10.1787/9789264073234-en
   Google Scholar
• Popham, W. J. (2007). Instructional insensitivity of tests: Accountability’s dire drawback. Phi Delta Kappan, 89(2), 146–155. https://doi.org/10.1177/003172170708900211
   Web of Science ®Google Scholar
• Popham, W. J., & Husek, T. R. (1969). Implications of criterion-referenced measurement. Journal of Educational Measurement, 6(1), 1–9. https://doi.org/10.1111/j.1745-3984.1969.tb00654.x
   Web of Science ®Google Scholar
• Redman, E. J. K. H., Suh, Y. S., Cai, L., Baker, E. L., Parks, C. B., & Chung, G. K. W. K. (2020). Feature analysis of Cat in the Hat Builds That app and assessments (Deliverable to EDC). UCLA/CRESST.
   Google Scholar
• Rossi, P. H., & Wright, J. D. (1984). Evaluation research: An assessment. Annual Review of Sociology, 10, 331–352. https://doi.org/10.1146/annurev.so.10.080184.001555
   Web of Science ®Google Scholar
• Scriven, M. (1967). The methodology of evaluation (AERA Monograph Series on Curriculum Evaluation No. 1). Rand McNally.
   Google Scholar
• Scriven, M. (1974). Evaluation perspectives and procedures. In W. J. Popham (Ed.), Evaluation in education. McCutchan Publishing.
   Google Scholar
• Spearman, C. (1904). “General intelligence,” objectively determined and measured. The American Journal of Psychology, 15(2), 201–292. https://doi.org/10.2307/1412107
   Google Scholar