Assessing the Development of Educational Research Literacy: The Effect of Courses on Research Methods in Studies of Educational Science

REFERENCES

• Adams, R.J. (2002). Scaling PISA cognitive data. In R.J. Adams & M.L. Wu (Eds.), PISA 2000 technical report (pp. 99–108). Paris, France: OECD.
   Google Scholar
• Altrichter, H., Feldman, A., Posch, P., & Somekh, B. (2008). Teachers investigate their work: An introduction to action research across the professions ( 2nd ed.). New York, NY: Routledge.
   Google Scholar
• Barchfeld, P., & Sodian, B. (2009). Differentiating theories from evidence: The development of argument evaluation abilities in adolescence and early adulthood. Informal Logic, 29(4), 396–416.
   Google Scholar
• Blixrud, J.C. (2003). Project SAILS: Standardized Assessment of Information Literacy Skills. ARL Bimonthly Report. Retrieved from http://www.arl.org/bm∼doc/sails.pdf
   Google Scholar
• Bloömeke, S., Gustafsson, J.-E., & Shavelson, R. (2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift für Psychologie, 3–13.
   Web of Science ®Google Scholar
• Blömeke, S., & Zlatkin-Troitschanskaja, O. (2013). Kompetenzmodellierung und Kompetenzerfassung im Hochschulsektor: Ziele, theoretischer Rahmen, Design und Herausforderungen des BMBF-Forschungsprogramms KoKoHs [Modeling and measuring competencies in higher education: Objectives, theoretical framework, design, and challenges of the BMBF research program KoKoHs] (KoKoHs Working Papers, 1). Retrieved from http://www.kompetenzen-im-hochschulsektor.de/Dateien/KoKoHs_WP1_Bloemeke_Zlatkin-Troitschanskaia_2013_.pdf
   Google Scholar
• Borg, S. (2010). Language teacher research engagement. Language Teaching, 43, 391–429.
   Web of Science ®Google Scholar
• Bromme, R., Prenzel, M., & Jäger, M. (2014). Empirische Bildungsforschung und evidenzbasierte Bildungspolitik. Eine Analyse von Anforderungen an die Darstellung, Interpretation und Rezeption empirischer Befunde [Educational research and evidence-based policy. An analysis of the requirements concerning the presentation, interpretation, and reception of empirical evidence]. Zeitschrift für Erziehungswissenschaft, Sonderheft, 27, 3–54.
   Web of Science ®Google Scholar
• Brown, N.J. S., Furtak, E.M., Timms, M., Nagashima, S.O., & Wilson, M. (2010). The evidence-based reasoning framework: Assessing scientific reasoning. Educational Assessment, 15(3/4), 123–141.
   Google Scholar
• Brown, N.J. S., Nagashima, S.O., Fu, A., Timms, M., & Wilson, M. (2010). A framework for analyzing scientific reasoning in assessments. Educational Assessment, 15, 142–174.
   Google Scholar
• Clanchy, J., & Ballard, B. (1995). Generic skills in the context of higher education. Higher Education Research and Development, 14(2), 155–166.
   Google Scholar
• Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.
   PubMed Web of Science ®Google Scholar
• Colquitt, J.A., & Zapata-Phelan, C.P. (2007). Trends in theory building and theory testing: A five-decade study of the Academy of Management Journal. Academy of Management Journal, 50(6), 1281–1303.
   Web of Science ®Google Scholar
• Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.
   PubMed Web of Science ®Google Scholar
• Dunlap, W.P., Cortina, J.M., Vaslow, J.B., & Burke, M.J. (1996). Meta-analysis of experiments with matched groups or repeated measures design. Psychological Methods, 1(2), 170–177.
   Web of Science ®Google Scholar
• Dunn, D.S., Halonen, J.S., & Smith, R.A. (Eds.). (2008). Teaching critical thinking in psychology: A handbook of best practices. Malden, MA: Wiley.
   Google Scholar
• Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.
   PubMed Web of Science ®Google Scholar
• Fischer, F., Wecker, C., Hetmanek, A., Osborne, J., Chinn, C.A., Duncan, R.G., … Sandoval, W.A. (2014). The interplay of domain-specific and domain-general factors in scientific reasoning and argumentation. Paper presented at the International Conference of the Learning Sciences: Boulder, Colorado.
   Google Scholar
• Flores-Mateo, G., & Argimon, J.M. (2007). Evidence based practice in postgraduate healthcare education: A systematic review. BMC health services research, 7, 119ff.
   PubMed Web of Science ®Google Scholar
• German Council of Science and Humanities. (2000). Empfehlungen zur Einführung neuer Studienstrukturen und -abschlüsse (Bakkalaureus/Bachelor—Magister/Master) in Deutschland [Recommendations for introducing new study structures and degrees (bachelor/master) in Germany]. Retrieved from http://www.wissenschaftsrat.de/download/archiv/4418-00.pdf
   Google Scholar
• Greiff, S., Kretzschmar, A., & Leutner, D. (2014). Problemlösen in der Pädagogischen Psychologie [Problem solving in educational psychology]. Zeitschrift für Pädagogische Psychologie, 28(4), 161–166.
   Web of Science ®Google Scholar
• Greiff, S., Wüstenberg, S., Holt, D., Goldhammer, F., & Funke, J. (2013). Computer-based assessment of complex problem solving: Concept, implementation, and application. Educational Technology Research & Development, 61(3), 407–421.
   Web of Science ®Google Scholar
• Groß Ophoff, J., Schladitz, S., Lohrmann, K., & Wirtz, M. (2014). Evidenzorientierung in bildungswissenschaftlichen Studiengängen: Entwicklung eines Strukturmodells zur Forschungskompetenz [Evidence-orientation in educational science degree programs: Development of a structure model of Educational Research Literacy]. In W. Bos, K. Drossel, & R. Strietholt (Eds.), Empirische Bildungsforschung und evidenzbasierte Reformen im Bildungswesen (pp. 251–276). Münster, Germany: Waxmann.
   Google Scholar
• Groß Ophoff, J., Wolf, R., & Haberfellner, C. (submitted). Evidence-based reasoning in higher education: Development and validation of a mixed-format test approach to assess Educational Research Literacy (KoKoHs Working papers, Vol. 8). Berlin and Mainz: Humboldt University and Johannes Gutenberg University.
   Google Scholar
• Groth, R.E. (2007). Toward a conceptualization of statistical knowledge for teaching. Journal for Research in Mathematics Education, 38(5), 427–437.
   Web of Science ®Google Scholar
• Grundmann, R., & Stehr, N. (2012). The power of scientific knowledge: From research to public policy: Cambridge, UK: Cambridge University Press.
   Google Scholar
• Hammersley, M. (2004). Some questions about evidence-based practice in education. In G. Thomas & R. Pring (Eds.), Evidence-based practice in education (Vol. 90, pp. 133–149). Maidenhead, UK: Open University Press.
   Google Scholar
• Hartig, J., & Höhler, J. (2009). Multidimensional IRT models for the assessment of competencies. Studies in Educational Evaluation, 35(2–3), 57–63.
   Google Scholar
• Hattie, J., & Timperley, H.S. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.
   Web of Science ®Google Scholar
• Kiefer, T., Robitzsch, A., & Wu, M. (2014). Test Analysis Modules (TAM) (Version 1.0-3.18-1). Retrieved from http://www.edmeasurementsurveys.com/TAM/Tutorials/
   Google Scholar
• Klein, S., Benjamin, R., Shavelson, R., & Bolus, R. (2007). The Collegiate Learning Assessment: Facts and fantasies. Evaluation Review, 31(5), 415–439.
   PubMed Web of Science ®Google Scholar
• Koeppen, K., Hartig, J., Klieme, E., & Leutner, D. (2008). Current issues in competence modeling and assessment. Zeitschrift für Psychologie, 216(2), 61–73.
   Web of Science ®Google Scholar
• König, J., Blömeke, S., Paine, L., Schmidt, W.H., & Hsieh, F.-J. (2011). General pedagogical knowledge of future middle school teachers: On the complex ecology of teacher education in the United States, Germany, and Taiwan. Journal of Teacher Education, 62(2), 188–201.
   Web of Science ®Google Scholar
• Kuhn, D., & Franklin, S. (2007). The second decade: What develops (and how), Handbook of Child Psychology (part VIII, pp. 517–550). Hoboken, NJ: John Wiley & Sons.
   Google Scholar
• Kuhn, D., Iordanou, K., Pease, M., & Wirkala, C. (2010). Beyond control of variables: What needs to develop to achieve skilled scientific thinking? Cognitive Development, 23(4), 435–451.
   Web of Science ®Google Scholar
• Lea, M. R. & Street, B. V. (2006). The “Academic Literacies” Model: Theory and Applications. Theory Into Practice, 45(4), 368–377
   Google Scholar
• Mayer, R.E., & Wittrock, M.C. (2006). Problem solving. In P.A. Alexander & P.H. Winne (Eds.), Handbook of educational psychology (pp. 287–303). . Mahwah, NJ: Lawrence Erlbaum.
   Google Scholar
• Meissner, D., Vogel, E., & Horn, K.-P. (2012). Lehrerausbildung in Baden-Württemberg seit 1945. Angleichungs- und Abgrenzungsprozesse [Teacher education in Baden-Wüerttemberg since 1945: Processes of alignment and demarcation]. In C. Cramer (Ed.), Lehrerausbildung in Baden-Württemberg. Historische Entwicklungslinien und aktuelle Herausforderungen (pp. 33–62). Jena, Germany: IKS Garamond.
   Google Scholar
• Ministry of Cultural Affairs of Baden-Wüerttemberg. (2011). Verordnung des Kultusministeriums uöber die Erste Staatspruöfung fuör das Lehramt an Grundschulen (Grundschullehramts­pruö­fungs­ord­nung I). GPO I [Regulation of the Ministry of Cultural Affairs of Baden-Württemberg on the first state examination for elementary school teaching]. Retrieved from http://www.landesrecht-bw.de/jportal/?quelle=jlink&docid=jlr-GHLehr1PrOBW2011rahmen&psml=bsbawueprod.psml&max=true
   Google Scholar
• Mittelhäuser, M.-A., Béguin, A.A., & Sijtsma, K. (2011). Comparing the effectiveness of different linking designs. The internal anchor versus the external anchor and pre-test data measurement and research department reports. Arnhem, Netherlands: Cito.
   Google Scholar
• National Association for the Early Education of Children. (2009). Qualifikationsrahmen für BA-Studiengänge der “Kindheitspädagogik”/“Bildung und Erziehung in der Kindheit” [Qualifications framework for Early Childhood Education B.A. degree programs]. Retrieved from http://www.ku.de/fileadmin/18/Praxis/BAG-BEK-BA-QR-final030110.pdf
   Google Scholar
• Newton, P.E., & Shaw, S.D. (2014). Validity in educational and psychological assessment. Los Angeles, CA: Cambridge Assessment, Sage.
   Google Scholar
• Novick, L.R., & Bassok, M. (2005). Problem solving. In K. Holyoak & B. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 321–349). New York, NY: Cambridge University Press.
   Google Scholar
• Phye, G.D. (2001). Problem-solving instruction and problem-solving transfer: The correspondence issue. Journal of Educational Psychology, 93(3), 571–578.
   Web of Science ®Google Scholar
• Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Nielsen & Lydiche.
   Google Scholar
• Rost, J. (2004). Lehrbuch Testtheorie/Testkonstruktion ( 2nd ed.) . [Course book test theory/test construction]. Bern, Germany: Huber.
   Google Scholar
• Rott, B., Leuders, T., & Stahl, E. (2015). Assessment of mathematical competencies and epistemic cognition of pre-service teachers. Zeitschrift für Psychologie, 223(1), 3946.
   Web of Science ®Google Scholar
• Schladitz, S., Groß Ophoff, J., & Wirtz, M. (2015). Forschungskompetenz und allgemeine Intelligenz. Ein Beitrag zur Konstruktvalidierung von Kompetenztests in bildungs wissen schaftlichen Studiengangen [Research literacy and intelligence: A contribution to construct validation of competency tests in educational science degree programs]. Zeitschrift für Pädagogik, 61. Beiheft, 167184.
   Google Scholar
• Schladitz, S., Rott, B., Winter, A., Wischgoll, A., Groß Ophoff, J., Hosenfeld, I., … Wittwer, J. (2013). LeScEd—Learning the Science of Education: Research competence in educational sciences. In S. Blömeke & O. Zlatkin-Troitschanskaja (Eds.), The German funding initiative “Modeling and Measuring Competencies in Higher Education” (pp. 82–84). Berlin and Mainz: Humboldt Universität and Johannes Gutenberg Universität.
   Google Scholar
• Schmid, S., Richter, T., Berthold, K., Bruns, K., & von der Mühlen, S. (2013). KOSWO—Students’ competencies when dealing with scientific primary literature. In S. Blömeke & O. Zlatkin-Troitschanskaja (Eds.), The German funding initiative “Modeling and Measuring Competencies in Higher Education” (pp. 71–74). Berlin and Mainz: Humboldt Universität and Johannes Gutenberg Universität.
   Google Scholar
• Shank, G., & Brown, L. (2007). Exploring Educational Research Literacy. New York, NY: Routledge.
   Google Scholar
• Spinath, B., Stiensmeier-Pelster, J., Schöne, C., & Dickhäuser, O. (2002). SELLMO. Skalen zur Erfassung der Lern- und Leistungsmotivation. Göttingen, Germany: Hogrefe.
   Google Scholar
• Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany. (2004). Standards für die Lehrerbildung: Bildungswissenschaften. Beschluss der Kultusministerkonferenz vom 16.12.2004 [Standards for teacher education: Educational science. Regulation of the Standing Conference of the Ministers of Education and Cultural Affairs from December 16, 2004]. Retrieved 2010, from http://www.kmk.org/fileadmin/veroeffentlichungen_beschluesse/2004/2004_12_16-Standards-Lehrerbildung.pdf
   Google Scholar
• Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany. (2005). Qualifikationsrahmen für deutsche Hochschulabschlüsse [Qualifications framework for German university degrees]. Retrieved from http://www.kmk.org/fileadmin/pdf/PresseUndAktuelles/Beschluesse_Veroeffentlichungen/Hochschule_Wissenschaft/BS_050421_Qualifikationsrahmen_AS_Ka.pdf
   Google Scholar
• Trempler, K. (2013). KOMPARE—Competent argumentation with evidences: Measurement and modeling in educational sciences and transfer from medical studies. In S. Blömeke & O. Zlatkin-Troitschanskaja (Eds.), The German funding initiative “Modeling and Measuring Competencies in Higher Education” (pp. 78–81). Berlin and Mainz: Humboldt Universität and Johannes Gutenberg Universität.
   Google Scholar
• University of Education Freiburg. (2009). Studien- und Prüfungsordnung der Pädagogischen Hochschule Freiburg für die Bachelor-Studiengänge Pädagogik der frühen Kindheit [Study and examination regulation of the University of Education Freiburg for Early Childhood Education B.A. degree program]. Retrieved from https://http://www.ph-freiburg.de/fileadmin/dateien/zentral/webdoks/hochschule/bekanntmachungen/ab_0917_spo_bachelor_pfk.pdf
   Google Scholar
• von Davier, M., Gonzalez, E., & Mislevy, R.J. (2009). What are plausible values and why are they useful? IERI Monograph Series, 4, 9–36.
   Google Scholar
• Watson, J.M., & Callingham, R.A. (2003). Statistical literacy: A complex hierarchical construct. Statistics Education Research Journal, 2(2), 3–46.
   Google Scholar
• Watt, H.M. G., Richardson, P.W., Klusmann, U., Kunter, M., Beyer, B., Trautwein, U., & Baumert, J. (2012). Motivations for choosing teaching as a career: An international comparison using the FIT-Choice scale. Teaching and Teacher Education, 28(6), 791–805.
   Web of Science ®Google Scholar
• Willison, J., & O’Regan, K. (2007). Commonly known, commonly not known, totally unknown: A framework for students becoming researchers. Higher Education Research & Development, 26(4), 393–409.
   Google Scholar
• Wright, D.B., & Douglas, G.A. (1996). Estimating Rasch (person, ability, theta) measures with known dichotomous item difficulties: Anchored maximum likelihood estimation (AMLE). Rasch Measurement Transactions Contents. Archives of the Rasch Measurement SIG, AERA, 10(2), 499ff.
   Google Scholar
• Wu, M. (2004). Plausible values. Rasch Measurement Transactions, 18(2), 976–978.
   Google Scholar
• Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2–3), 114–128.
   Google Scholar
• Wu, M.L., Adams, R.J., Wilson, M.R., & Haldane, S. (2001). ConQuest (Version 2.0). Camberwell, Victoria: Australian Council for Educational Research Ltd (ACER).
   Google Scholar