Measuring Students’ Writing Ability on a Computer-Analytic Developmental Scale: An Exploratory Validity Study

References

• ACT. (2010). The condition of college and career readiness 2010. Iowa City, IA: ACT. http://www.act.org/research/policymakers/cccr10/pdf/ConditionofCollegeandCareerReadiness2010.pdf ( http://www.act.org/research/policymakers/cccr10/pdf/ConditionofCollegeandCareerReadiness2010.pdf)
   Google Scholar
• Alamargot , D. and Fayol , M. 2009 . “ Modelling [sic] the development of written composition ” . In The SAGE handbook of writing development , Edited by: Beard , R. , Myhill , D. , Riley , J. and Nystrand , M. 23 – 47 . Thousand Oaks , CA : SAGE .
   Google Scholar
• Applebee , A. N. , Langer , J. I. and Mullis , I. V. 1986 . The writing report card: Writing achievement in American schools. (Report number 15-W-02) , Princeton , NJ : Educational Testing Service .
   Google Scholar
• American Educational Research Association, American Psychological Association and National Council on Measurement in Education . 1999 . The standards for educational and psychological testing , Washington , DC : American Educational Research Association, American Psychological Association, National Council on Measurement in Education .
   Google Scholar
• Attali , Y. and Burstein , J. 2006 . Automated essay scoring with E-rater V. 2.0 . The Journal of Technology, Learning, and Assessment , 4 ( 3 ) : 13 – 18 .
   Google Scholar
• Ben-Simon, A., & Bennett, R. L. (2007). Toward more substantively meaningful automated essay scoring. The Journal of Technology, Learning, and Assessment, 6. http://www.jtla.org (http://www.jtla.org)
   Google Scholar
• Bereiter , C. and Scardamalia , M. 1987 . The psychology of written composition , Hillsdale , NJ : Erlbaum .
   Google Scholar
• Betts , E. A. 1946 . Foundations of reading instruction with emphasis on differentiated guidance , New York , NY : American Book Company .
   Google Scholar
• Bill and Melinda Gates Foundation. (n.d.). Request for proposal: Literacy courseware challenge. http://www.gatesfoundation.org/grantseeker/Documents/Literacy-Courseware-Challege-RFP.pdf (http://www.gatesfoundation.org/grantseeker/Documents/Literacy-Courseware-Challege-RFP.pdf)
   Google Scholar
• Bond , T. G. and Fox , C. M. 2007 . Fundamental measurement in the human sciences , 2nd , Mahwah , NJ : Erlbaum .
   Google Scholar
• Breland , H. M. , Camp , R. , Jones , R. J. , Morris , M. M. and Rock , D. A. 1987 . Assessing writing skill , New York , NY : College Entrance Examination Board . Research Monograph No. 11)
   Google Scholar
• Brennan, R. L. (2001). Generalizability theory. New York, NY: Springer-Verlag. For mGENOVA. Retrieved from http://www.education.uiowa.edu/casma/GenovaPrograms.htm (http://www.education.uiowa.edu/casma/GenovaPrograms.htm)
   Google Scholar
• Chenoweth , N. A. and Hayes , J. R. 2001 . The inner voice in writing . Written Communication , 20 : 99 – 118 .
   Web of Science ®Google Scholar
• Chipere, N., Malvern, D. D., Richards, B. J., & Durán, P. (2001). Using a corpus of school children's writing to investigate the development of lexical diversity. In P. Rayson, A. Wilson, T. McEnery, A. Hardie, & S. Khoja (Eds.), Proceedings of the Corpus Linguistics 2001 Conference (pp. 126–133). Lancaster, UK: University Centre for Computer Corpus Research on Language. http://ucrel.lancs.ac.uk/publications/CL2003/CL2001%20conference/papers/chipere.pdf (http://ucrel.lancs.ac.uk/publications/CL2003/CL2001%20conference/papers/chipere.pdf)
   Google Scholar
• Cohen , J. 1988 . Statistical power analysis for the behavioral sciences , Hillsdale , NJ : Erlbaum .
   Google Scholar
• Cope , B. , Kalantzis , M. , McCarthey , S. , Vojak and Kline , S. 2011 . “ Technology-mediated writing assessments: Principles and processes ” . In Computers and Composition Vol. 28 , 79 – 96 .
   Google Scholar
• Crosson , B. , Rao , S. M. , Woodley , S. J. , Rosen , A. C. , Hammeke , T. A. , Bobholz , J. A. … and Stein , E. A. 1999 . Mapping of semantic, phonological, and orthographic verbal working memory in normal adults with functional magnetic resonance imaging . Neuropsychology , 13 : 171 – 187 .
   PubMed Web of Science ®Google Scholar
• Deane , P. 2013 . On the relation between automated essay scoring and modern views of the writing construct . Assessing Writing , 18 : 7 – 24 .
   Web of Science ®Google Scholar
• Deane , P. and Quinlan , T. 2010 . What automated analyses of corpora can tell us about student's writing . Journal of Writing Research , 2 : 151 – 177 .
   Google Scholar
• Donovan , C. A. and Smolkin , L. B. 2006 . “ Children's understanding of genre and writing development ” . In Handbook of writing research , Edited by: MacArthur , C. A. , Graham , S. and Fitzgerald , J. 131 – 143 . New York , NY : Guilford .
   Google Scholar
• Duncan , O. D. 1968 . Ability and achievement . Eugenics Quarterly , 15 : 1 – 11 .
   PubMed Web of Science ®Google Scholar
• Elliot , N. 2005 . On a scale: A social history of writing assessment in America , New York , NY : Peter Lang .
   Google Scholar
• Freebody , P. and Anderson , R. C. 1983 . Effects on text comprehension of differing proportions and locations of difficult vocabulary . Journal of Reading Behavior , 15 : 19 – 39 .
   Web of Science ®Google Scholar
• García , G. E. 1991 . Factors influencing the English reading test performance of Spanish-speaking Hispanic children . Reading Research Quarterly , 26 : 371 – 392 .
   Web of Science ®Google Scholar
• Goodman , K. S. 1967 . Reading: A psycholinguistic guessing game . Journal of the Reading Specialist , 6 : 126 – 135 .
   Web of Science ®Google Scholar
• Graham , S. and Harris , K. 2011 . “ Writing difficulties ” . In Handbook of reading disability research , Edited by: McGill-Franzen , A. and Allington , R. L. 232 – 241 . New York , NY : Routledge .
   Google Scholar
• Graham , S. and Perin , D. 2007 . Writing next: Effective strategies to improve writing of adolescents in middle and high schools , New York , NY : Carnegie Corporation of New York .
   Google Scholar
• Graham , S. , Harris , K. and Hebert , M. 2011a . Informing writing: The benefits of formative assessment: A report from Carnegie Corporation of New York , New York , NY : Carnegie Corporation .
   Google Scholar
• Graham , S. , Harris , K. R. and Hebert , M. 2011b . It is more than just the message: Presentation effects in scoring writing . Focus on Exceptional Children , 44 ( 4 ) : 1 – 12 .
   Web of Science ®Google Scholar
• Graham , S. , Hebert , M. and Harris , K. R. 2011 . Throw ‘em out or make ‘em better? State and district high-stakes writing assessments . Focus on Exceptional Children , 44 ( 1 ) : 1 – 12 .
   Web of Science ®Google Scholar
• Harris , K. R. , Graham , S. , Brindle , M. and Sandmel , K. 2009 . “ Metacognition and children's writing ” . In Handbook of metacognition in education , Edited by: Hacker , D. , Dunlosky , J. and Graesser , A. 131 – 153 . Mahwah , NJ : Erlbaum .
   Google Scholar
• Hudson , R. 2009 . “ Measuring maturity ” . In The SAGE handbook of writing development , Edited by: Beard , R. , Myhill , D. , Riley , J. and Nystrand , M. 349 – 362 . London , , UK : SAGE .
   Google Scholar
• Hunter , J. E. and Schmidt , F. L. 2004 . Methods of meta-analysis: Correcting error and bias in research findings , Thousand Oaks , CA : Sage .
   Google Scholar
• Jenkins , J. R. , Johnson , E. and Hileman , J. 2004 . When is reading also writing: Sources of individual differences on the new reading performance assessments . Scientific Studies of Reading , 8 : 125 – 151 .
   Web of Science ®Google Scholar
• Johns , J. L. 2005 . Basic reading inventory: Preprimer through grade twelve and early literacy assessments , Dubuque , IA : Kendall/Hunt .
   Google Scholar
• Jurafsky , D. and Martin , J. 2000 . Speech and language processing , Englewood , NJ : Prentice Hall .
   Google Scholar
• Kasson , S. 2003 . Psychology , Upper Saddle River , NJ : Prentice-Hall .
   Google Scholar
• Kellogg , R. T. 2008 . Training writing skills: A cognitive developmental perspective . Journal of Writing Research , 1 : 1 – 26 .
   Google Scholar
• Kolen , M. May 2010 . Update on the revision of the standards for educational and psychology assessment: Overview May , Denver , CO Presentation at the annual meeting of the National Council on Measurement in Education
   Google Scholar
• Kolen , M. J. 2004 . Population invariance in equating and linking: Concept and history . Journal of Educational Measurement , 41 : 3 – 14 .
   Web of Science ®Google Scholar
• Landauer , T. K. , Foltz , D. and Laham , D. 1998 . Introduction to latent semantic analysis . Discourse Processes , 25 : 259 – 284 .
   Web of Science ®Google Scholar
• Linacre , J. M. 1994 . Many-facet Rasch measurement , Chicago , IL : MESA .
   Google Scholar
• Linacre , J. M. 2010 . A user's guide to FACETS: Rasch-model computer programs , Chicago , IL : Winsteps.com .
   Google Scholar
• Linacre , J. M. and Wright , B. D. 2002 . Construction of measures from many-facet data . Journal of Applied Measurement , 3 : 486 – 512 .
   PubMedGoogle Scholar
• Luke , A. 1988 . Literacy, textbooks, and ideology , Lewes , , UK : Falmer Press .
   Google Scholar
• Marcus , M. P. , Marcinkiewicz , M. A. and Santorini , B. 1993 . Building a large annotated corpus of English: The Penn Treebank . Computational Linguisitics , 19 : 313 – 330 .
   Google Scholar
• McCutchen , D. 1996 . A capacity theory of writing: Working memory in composition . Educational Psychology Review , 8 : 299 – 325 .
   Web of Science ®Google Scholar
• National Center for Education Statistics. (2008). The nation's report card: Writing 2007. Washington, DC: National Center for Education Statistics, Institute for Education Sciences, U. S. Office of Education. http://nces.ed.gov/nationsreportcard/writing/ (http://nces.ed.gov/nationsreportcard/writing/)
   Google Scholar
• National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for English language arts & literacy in history/social studies, science, and technical subjects, Appendix A. Washington, DC: Author. http://www.corestandards.org/the-standards (http://www.corestandards.org/the-standards)
   Google Scholar
• Page , E. B. 1994 . Computer grading of student prose, using modern concepts and software . Journal of Experimental Education , 62 : 127 – 142 .
   Web of Science ®Google Scholar
• Page , E. B. 1996 . The imminence of grading essays by computer . Phi Delta Kappan , 48 : 238 – 243 .
   Google Scholar
• Partnership for Assessment of Readiness for College and Careers. (n.d.). Item and task prototypes. http://www.parcconline.org/samples/item-task-prototypes (http://www.parcconline.org/samples/item-task-prototypes)
   Google Scholar
• Penn Treebank Project. (n.d.). www.cis.upenn.edu/˜treebank/ (http://www.cis.upenn.edu/~treebank/)
   Google Scholar
• Perfetti , C. A. 1998 . “ Two basic questions about reading and learning to read ” . In Problems and interventions in literacy development , Edited by: Reitsma , P. and Verhoeven , L. 1 – 48 . Dordrecht , , Netherlands : Kluwer .
   Google Scholar
• Proctor , C. P. , Carlo , M. , August , D. and Snow , C. 2005 . Native Spanish-speaking children reading in English: Toward a model of comprehension . Journal of Educational Psychology , 97 : 246 – 256 .
   Web of Science ®Google Scholar
• Python Toolkit. (n.d.). http://pythontoolkit.sourceforge.net (http://pythontoolkit.sourceforge.net)
   Google Scholar
• Rasch , G. 1960/1980 . Probabilistic models for some intelligence and attainment tests , Copenhagen , , Denmark : Danish Institute for Educational Research . Expanded edition, 1980. Chicago, IL: University of Chicago Press
   Google Scholar
• Sainsbury , M. 2009 . “ Developing writing in a high-stakes environment ” . In The SAGE handbook of writing development , Edited by: Beard , R. , Myhill , D. , Riley , J. and Nystrand , M. 545 – 560 . Los Angeles , CA : SAGE .
   Google Scholar
• SAT. (n.d.). http://sat.collegeboard.com/home (http://sat.collegeboard.com/home)
   Google Scholar
• Shen , W. , Kiger , T. B. , Davies , S. E. , Rasch , R. L. , Simon , K. M. and Ones , D. S. 2011 . Samples in applied psychology: Over a decade of research in review . Journal of Applied Psychology , 96 : 1055 – 1064 .
   PubMed Web of Science ®Google Scholar
• Shermis , M. D. and Burstein , J. 2003 . Automated essay scoring: A cross-disciplinary perspective , Hillsdale , NJ : Erlbaum .
   Google Scholar
• Shermis , M. D. , Burstein , J. and Leacock , C. 2006 . “ Applications of computers in assessment and analysis of writing ” . In Handbook of writing research , Edited by: MacAthur , C. A. , Graham , S. and Fitzgerald , J. 403 – 416 . New York , NY : Guilford .
   Google Scholar
• Smarter Balanced Assessment Consortium. (n.d.). Smarter balance assessments. Retrieved from http://www.smarterbalanced.org/smarter-balanced-assessments/ (http://www.smarterbalanced.org/smarter-balanced-assessments/)
   Google Scholar
• Stenner , A. J. , Burdick , D. and Fitzgerald , J. 2012 . Text complexity: Toward construct definition and measurement July)Paper presentation in a symposium on text complexity at the annual meeting of the Scientific Study for Reading, Montreal, Canada
   Google Scholar
• Sturm , J. M. 2012 . “ An enriched writers’ workshop approach: Implementation of high-quality, differentiated writing instruction for a range of learners ” . In Topics in Language Disorders 335 – 360 . 32
   Google Scholar
• Taylor , S. E. , Frackenpohl , H. and White , C. E. 1979 . EDL core vocabularies in reading, mathematics, science, and social studies , New York , NY : McGraw-Hill .
   Google Scholar
• U.S. Department of Education, Institute of Education Sciences. (1998). Sample questions: Writing 1998. Washington, DC: National Center for Education Statistics. http://nces.ed.gov/nationsreportcard/ (http://nces.ed.gov/nationsreportcard/)
   Google Scholar
• U.S. Department of Education, Institute of Education Sciences. (2002). Sample questions: Writing 2002. Washington, DC: National Center for Education Statistics. http://nces.ed.gov/nationsreportcard/ (http://nces.ed.gov/nationsreportcard/)
   Google Scholar
• U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2011). The nation's report card; Writing 2011: National assessment of educational progress at grades 8 and 12. Washington, DC: Institute of Education Sciences, National Center for Education Statistics, U. S. Department of Education. http://nces.ed.gov/nationsreportcard/writing/ (http://nces.ed.gov/nationsreportcard/writing/)
   Google Scholar
• Vanderberg , R. and Swanson , H. L. 2007 . Which components of working memory are important in the writing process? . Reading and Writing , 20 : 721 – 752 .
   Web of Science ®Google Scholar
• Verhoeven , L. T. 2000 . Components in early second language reading and spelling . Scientific Studies of Reading , 4 : 313 – 330 .
   Google Scholar
• Vojak , C. , Kline , S. , Cope , B. , McCarthey , S. and Kalantzis , M. 2011 . New spaces and old places: An analysis of writing assessment software . Computers and Composition , 28 : 97 – 111 .
   Google Scholar
• Wagner , D. H. and Venezky , R. L. 1999 . Adult literacy: The next generation . Educational Researcher , 28 : 21 – 29 .
   Google Scholar
• Wagner , Q. , Paré-Blagoev , E. J. , Clark , J. and Poldrack , R. 2001 . Recovering meaning: Left prefrontal cortex guides controlled semantic retrieval . Neuron , 31 : 329 – 338 .
   PubMed Web of Science ®Google Scholar
• Wallace, S. (Ed.). (2009). Dictionary of education. Oxford Reference Online. Oxford University Press. www.oxfordreference.com.libproxy.lib.unc.edu/views/ENTRY.html?subview=Main&entry=t267.e3> (http://www.oxfordreference.com.libproxy.lib.unc.edu/views/ENTRY.html?subview=Main&entry=t267.e3>)
   Google Scholar