Does Test Item Performance Increase with Test-to-Standards Alignment?

References

• Abedi, J. (2004). The No Child Left Behind Act and English language learners: Assessment and accountability issues. Educational Researcher, 33(1), 4–14.
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
   Google Scholar
• Anderson, D., Irvin, S., Alonzo, J., & Tindal, G. A. (2015). Gauging item alignment through online systems while controlling for rater effects. Educational Measurement: Issues and Practice, 34(1), 22–33.
   Web of Science ®Google Scholar
• Bhola, D. J., Impara, J. C., & Buckendahl, C. W. (2003). Aligning tests with states’ content standards: Methods and issues. Educational Measurement: Issues and Practice, 22(3), 21–29.
   Google Scholar
• Blank, R. K., & Smithson, J. (2009). Alignment content analysis of TIMSS and PISA Mathematics and Science assessments using the Surveys of Enacted Curriculum methodology. Washington, DC: Council of Chief State School Officers.
   Google Scholar
• Buckendahl, C. W., Davis, S. L., Plake, B. S., Sireci, S. G., Hambleton, R. K., Zenisky, A. L., & Wells, C. S. (2009). Evaluation of the national assessment of educational progress: Final report. Washington, DC: U.S. Department of Education.
   Google Scholar
• Carmichael, S. B., Martino, G., Porter-Magee, K., & Wilson, W. S. (2010). The state of state standards—And the Common Core—In 2010. Washington, DC: Thomas B. Fordham Institute.
   Google Scholar
• Crocker, L. M., Miller, M. D., & Franks, E. A. (1989). Quantitative methods for assessing the fit between test and curriculum. Applied Measurement in Education, 2(2), 179–194.
   Google Scholar
• D’Agostino, J. V., Welsh, M. E., Cimetta, A. D., Falco, L. D., Smith, S., VanWinkle, W. H., & Powers, S. J. (2008). The rating and matching item-objective alignment methods. Applied Measurement in Education, 21(1), 1–21.
   Web of Science ®Google Scholar
• D’Agostino, J. V., Welsh, M. E., & Corson, N. M. (2007). Instructional sensitivity of a state’s standards-based assessments. Educational Assessment, 12(1), 1–22.
   Google Scholar
• Davis-Becker, S. L., & Buckendahl, C. W. (2013). A proposed framework for evaluating alignment studies. Educational measurement: Issues and practice, 32(1), 23–33.
   Web of Science ®Google Scholar
• de Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York, NY: Springer.
   Google Scholar
• Embretson, S. E., & Daniel, R. C. (2008). Understanding and quantifying cognitive complexity level in mathematical problem solving items. Psychological Science Quarterly, 50(3), 328–344.
   Google Scholar
• Enright, M. K., & Sheehan, K. M. (2002). Modeling the difficulty of quantitative reasoning items: Implications for item generation. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 129–157). Mahweh, NJ: Erlbaum.
   Google Scholar
• Ferrara, S., Svetina, D., Skucha, S., & Davidson, A. H. (2011). Test design with performance standards and achievement growth in mind. Educational Measurement: Issues and Practice, 30(4), 3–15.
   Google Scholar
• Gamoran, A., Porter, A. C., Smithson, J., & White, P. A. (1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19(4), 325–338.
   Web of Science ®Google Scholar
• Gottfried, M. A. (2009). Excused versus unexcused: How student absences in elementary school affect academic achievement. Educational Evaluation and Policy Analysis, 31(4), 392–415.
   Web of Science ®Google Scholar
• Harwell, M., & LeBeau, B. (2010). Student eligibility for a free lunch as an SES measure in education research. Educational Researcher, 39(2), 120–131.
   Web of Science ®Google Scholar
• Immekus, J. C., & Atitya, B. (2016). The predictive validity of interim assessment scores based on the full-information bifactor model for the prediction of end-of-grade test performance. Educational Assessment, 21(3), 176–195.
   Web of Science ®Google Scholar
• Jackson, C. K., Johnson, R. C., & Persico, C. (2016). The effects of school spending on educational and economic outcomes: Evidence from school finance reforms. Quarterly Journal of Economics, 131(1), 157–218.
   Web of Science ®Google Scholar
• Jordan, N. C., Hanich, L. B., & Kaplan, D. (2003). A longitudinal study of mathematical competencies in children with specific mathematics difficulties versus children with comorbid mathematics and reading difficulties. Child Development, 74(3), 834–850.
   PubMed Web of Science ®Google Scholar
• Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.
   Web of Science ®Google Scholar
• Koretz, D. (2008). Further steps toward the development of an accountability-oriented science of measurement. In K. E. Ryan & L. A. Shepard (Eds.), The future of test-based educational accountability (pp. 71–91). New York, NY: Routledge.
   Google Scholar
• Lee, J. S., & Bowen, N. K. (2006). Parent involvement, cultural capital, and the achievement gap among elementary school children. American Educational Research Journal, 43(2), 193–218.
   Web of Science ®Google Scholar
• Leinhart, G., & Seewald, A. M. (1981). Overlap: What’s tested, what’s taught? Journal of Educational Measurement, 18(2), 85–96.
   Web of Science ®Google Scholar
• Li, H., Qin, Q., & Lei, P. W. (2016). An examination of the instructional sensitivity of the TIMSS math items: A hierarchical differential item functioning approach. Educational Assessment, Advance online publication. doi:10.1080/10627197.2016.1271702
   Web of Science ®Google Scholar
• Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316.
   Web of Science ®Google Scholar
• Martineau, J., Paek, P., Keene, J., & Hirsch, T. (2007). Integrated, comprehensive alignment as a foundation for measuring student progress. Educational Measurement: Issues and Practice, 26(1), 28–35.
   Google Scholar
• Martone, A., & Sireci, S. G. (2009). Evaluating alignment between curriculum, assessment, and instruction. Review of Educational Research, 79(4), 1332–1361.
   Web of Science ®Google Scholar
• McNeish, D., Stapleton, L. M., & Silverman, R. D. (in press). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods. doi:10.1037/met0000078
   Web of Science ®Google Scholar
• Mehrens, W. A., & Phillips, S. E. (1986). Detecting impacts of curricular differences in achievement test data. Journal of Educational Measurement, 23(3), 185–196.
   Web of Science ®Google Scholar
• Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2002). On the roles of task model variables in assessment design. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 97–128). Mahweh, NJ: Erlbaum.
   Google Scholar
• Mullis, I. V., Martin, M. O., Ruddock, G. J., O’Sullivan, C. Y., Arora, A., & Erberber, E. (2005). TIMSS 2007 Assessment Frameworks. Chestnut Hill, MA: Boston College, Lynch School of Education, TIMSS & PIRLS International Study Center.
   Google Scholar
• Muthén, B. O., Kao, C.-F., & Burstein, L. (1991). Instructionally-sensitive psychometrics: Application of a new IRT-based detection technique to mathematics test items. Journal of Educational Measurement, 28(1), 1–22.
   Web of Science ®Google Scholar
• Naumann, A., Hochweber, J., & Klieme, E. (2016). A psychometric framework for the evaluation of instructional sensitivity. Educational Assessment, 21(2), 89–101.
   Web of Science ®Google Scholar
• Newton, J. A., & Kasten, S. E. (2013). Two models for evaluating alignment between state standards and assessments: Competing or complementary perspectives. Journal for Research in Mathematics Education, 44(3), 550–580.
   Web of Science ®Google Scholar
• Phillips, S. E., & Mehrens, W. A. (1988). Effects of curricular differences on achievement test data at item and objective levels. Applied Measurement in Education, 1(1), 33–51.
   Google Scholar
• Polikoff, M. S. (2012). Instructional alignment under No Child Left Behind. American Journal of Education, 118(3), 341–368.
   Web of Science ®Google Scholar
• Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3–14.
   Google Scholar
• Porter, A. C., Polikoff, M. S., Zeidner, T., & Smithson, J. (2008). The quality of content analyses of state student achievement tests and state content standards. Educational Measurement: Issues and Practice, 27(4), 2–14.
   Google Scholar
• Provasnik, S., KewalRamani, A., Coleman, M. M., Gilbertson, L., Herring, W., & Xie, Q. (2007). Status of education in rural America (NCES 2007-040). Washington, DC: National Center for Education Statistics.
   Google Scholar
• Raykov, T. (2012). Scale construction and development using structural equation modeling. In R. Hoyle (Ed.), Handbook of structural equation modeling (pp. 472–492). New York, NY: Guilford.
   Google Scholar
• Reardon, S. F., & Galindo, C. (2009). The Hispanic-White achievement gap in math and reading in the elementary grades. American Educational Research Journal, 46(3), 853–891.
   Web of Science ®Google Scholar
• Resnick, L. B., Rothman, R., Slattery, J. B., & Vranek, J. L. (2004). Benchmarking and alignment of standards and testing. Educational Assessment, 9(1&2), 1–27.
   Google Scholar
• Reys, B., Chval, K., Dingman, S., McNaught, M., Regis, T. P., & Togashi, J. (2007). Grade-level learning expectations: A new challenge for elementary mathematics teachers. Teaching Children Mathematics, 14(1), 6–11.
   Google Scholar
• Ruiz-Primo, M. A., Li, M., Wills, K., Giamellaro, M., Lan, M. C., Mason, H., & Sands, D. (2012). Developing and evaluating instructionally sensitive assessments in science. Journal of Research in Science Teaching, 49(6), 691–712.
   Web of Science ®Google Scholar
• Schafer, W. D., Wang, J., & Wang, V. (2009). Validity in action: State validity evidence for compliance with NCLB. In R. W. Lissitz (Ed.), The concept of validity (pp. 173–193). Charlotte, NC: Information Age.
   Google Scholar
• Schmidt, W. H., & Houang, R. T. (2012). Curricular coherence and the Common Core State Standards for Mathematics. Educational Researcher, 41(8), 294–308.
   Web of Science ®Google Scholar
• Schmidt, W. H., & Maier, A. (2009). Opportunity to learn. In G. Sykes, B. Schneider, & D. N. Plank (Eds.), Handbook of education policy research (pp. 541–559). New York NY: Routledge.
   Google Scholar
• Shaftel, J., Belton-Kocher, E., Glasnapp, D., & Poggio, J. (2006). The impact of language characteristics in mathematics test items on the performance of English language learners and students with disabilities. Educational Assessment, 11(2), 105–126.
   Google Scholar
• Sireci, S. G. (1998). Gathering and analyzing content validity data. Educational Assessment, 5(4), 299–321.
   Google Scholar
• Stecher, B. M., Epstein, S., Hamilton, L. S., Marsh, J. A., Robyn, A., McCombs, J. S., & Naftel, S. (2008). Pain and gain: Implementing No Child Left Behind in three states, 2004–2006. Santa Monica, CA: RAND.
   Google Scholar
• U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics [NCES]. (2008). Digest of education statistics (2008 ed.) [Statistical tables]. Washington, DC: Author. Retrieved from http://nces.ed.gov/programs/digest/2008menu_tables.asp
   Google Scholar
• U.S. Department of Education, National Assessment Governing Board [NAGB]. (2006, September). Mathematics framework for the 2007 National Assessment of Educational Progress. Washington, DC: Author.
   Google Scholar
• University of Wisconsin, Wisconsin Center for Educational Research, Measures of Enacted Curriculum Group [MECG]. (2010, November 9). Mathematics content analysis [Data files]. Retrieved from http://seconline.wceruw.org/MSP/Content/ELA/ELACntRpt/WSELACntRptMenu.asp
   Google Scholar
• Voight, A., Shinn, M., & Nation, M. (2012). The longitudinal effects of residential mobility on the academic achievement of urban elementary and middle school students. Educational Researcher, 41(9), 385–392.
   Web of Science ®Google Scholar
• Webb, N. L. (2007). Issues related to judging the alignment of curriculum standards and assessments. Applied Measurement in Education, 20(1), 7–25.
   Web of Science ®Google Scholar
• Wells, C. S., Baldwin, S., Hambleton, R. K., Sireci, S. G., Karatonis, A., & Jirka, S. (2009). Evaluating score equity assessment for State NAEP. Applied Measurement in Education, 22(4), 394–408.
   Web of Science ®Google Scholar
• Williams, R. T., Swanlund, A., Miller, S., Konstantopoulos, S., Eno, J., Van Der Ploeg, A., & Meyers, C. (2014). Measuring instructional differentiation in a large-scale experiment. Educational and Psychological Measurement, 74(2), 263–279.
   Web of Science ®Google Scholar
• Winfield, L. F. (1993). Investigating test content and curriculum content overlap to assess opportunity to learn. Journal of Negro Education, 62(3), 288–310.
   Google Scholar
• Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data (2nd ed.). Cambridge, MA: MIT Press.
   Google Scholar
• Zheng, B., & Agresti, A. (2000). Summarizing the predictive power of a generalized linear model. Statistics in Medicine, 19(13), 1771–1781.
   PubMed Web of Science ®Google Scholar