Vignette question design for the assessment of graduate sustainability learning outcomes

References

• Allen, J., and R. Van der Velden. 2005. “The Role of Self-assessment in Measuring Skills.” In REFLEX Working Paper 2, Research Centre for Education and the Labor Market. [Online] Accessed May 1, 2015. https://cris.maastrichtuniversity.nl/portal/files/4047131/guid-3b801703-f49b-46e5-bd15-adfe7eef189e-ASSET1.0
   Google Scholar
• Barrie, S. 2004. “A Research Based Approach to Generic Graduate Attributes Policy.” Higher Education Research and Development 23 (3): 261–275. Accessed Decemeber 1, 2016. https://sydney.edu.au/education-portfolio/ei/graduateAttributes/barriepaper1.pdf
   Google Scholar
• Barth, M., J. Godemann, M. Rieckmann, and U. Stoltenberg. 2007. “Developing Key Competencies for Sustainable Development in Higher Education.” International Journal of Sustainability in Higher Education 8 (4): 416–430.10.1108/14676370710823582
   Google Scholar
• Barth, M., and G. Michelsen. 2013. “Learning for Change: An Educational Contribution to Sustainability Science.” Sustainability Science 8 (1): 103–119.10.1007/s11625-012-0181-5
   Web of Science ®Google Scholar
• Bath, D., C. Smith, S. Stein, and R. Swann. 2004. “Beyond Mapping and Embedding Graduate Attributes: Bringing Together Quality Assurance and Action Learning to Create a Validated and Living Curriculum.” Higher Education Research & Development 23 (3): 313–328.10.1080/0729436042000235427
   Google Scholar
• Biggs, J., and C. Tang. 2007. Teaching for Quality Learning at University: What the Student Does. Maidenhead: Open University Press & McGraw-Hill.
   Google Scholar
• Bloor, M., and F. Wood. 2006. Keywords in Qualitative Methods. SAGE [Online]. Accessed December 1, 2016. http://dx.doi.org/10.4135/978184920940310.4135/9781849209403
   Google Scholar
• Bradburn, N., S. Sudman, and B. Wansink. 2004. Asking Questions: The Definitive Guide to Questionaire Design – for Market Research, Political Polls, and Social and Health Questionaires. San Francisco, CA: Jossey-Bass.
   Google Scholar
• Brundiers, K., A. Wiek, and C. L. Redman. 2010. “Real-world Learning Opportunities in Sustainability: From Classroom into the Real World.” International Journal of Sustainability in Higher Education 11 (4): 308–324.10.1108/14676371011077540
   Web of Science ®Google Scholar
• Buissink-Smith, N., S. Mann, and K. Shephard. 2011. “How Do We Measure Affective Learning in Higher Education?” Journal of Education for Sustainable Development 5 (1): 101–114.10.1177/097340821000500113
   Google Scholar
• Carteron, J., K. Haynes, and A. Murray. 2014. “Education for Sustainable Development, the Ungc Prme Initiative, and the Sustainability Literacy Test: Measuring and Assessing Success.” SAM Advanced Management Journal 79(4). 51–58.
   Google Scholar
• Cebrián, G., and M. Junyent. 2015. “Competencies in Education for Sustainable Development: Exploring the Student Teachers’ Views.” Sustainability 7 (3): 2768–2786.10.3390/su7032768
   Web of Science ®Google Scholar
• Clark, R. C., and R. E. Mayer. 2013. Scenario-based E-learning: Evidence-Based Guidelines for Online Workforce Learning. Wiley and Sons [Online]. Accessed December 5, 2016. http://rmit.eblib.com.au/patron/FullRecord.aspx?p=1097790
   Google Scholar
• Cohen, B. 2007. “Developing Educational Indicators that Will Guide Students and Institutions Toward a Sustainable Future.” New Directions for Institutional Research 134. 83–94.10.1002/(ISSN)1536-075X
   Google Scholar
• Coyle, K. 2005. Environmental Literacy in America. The National Environmental Education Training Foundation [Online]. Accessed December 5, 2016. https://www.neefusa.org/file/234/download?token=6Myt-Q55
   Google Scholar
• Dale, A., and L. Newman. 2005. “Sustainable Development, Education and Literacy.” International Journal of Sustainability in Higher Education 6 (4): 351–362.10.1108/14676370510623847
   Google Scholar
• Delaney, A. M. 2004. “Ideas to Enhance Higher Education’s Impact on Graduates’ Lives; Alumni Recommendations.” Tertiary Education and Management 10 (2): 89–105.10.1080/13583883.2004.9967120
   Google Scholar
• Dunlap, R. E., K. D. V. Liere, A. G. Mertig, and R. E. Jones. 2000. “New Trends in Measuring Environmental Attitudes: Measuring Endorsement of the New Ecological Paradigm: A Revised NEP Scale.” Journal of Social Issues 56 (3): 425–442.10.1111/0022-4537.00176
   Web of Science ®Google Scholar
• Finch, J. 1987. “The Vignette Technique in Survey Research.” Sociology 21 (1): 105–114.10.1177/0038038587021001008
   Google Scholar
• Fisher, F. 2006. Response Ability: Environment, Health and Everyday Transcendence. Elsterniwick: Vista.
   Google Scholar
• Franzosi, R. 2004. “Content Analysis.” In The Sage Encyclopedia of Social Science Research Methods, edited by M. S. Lewis-Beck, A. Bryman, and T. F. Liao, 187–191. Thousand Oaks, CA: Sage.
   Google Scholar
• de Haan, G. 2006. “The BLK ‘21’ programme in Germany: A ‘Gestaltungskompetenz’ - Based Model for Education for Sustainable Development.” Environmental Education Research 12 (1): 19–32.10.1080/13504620500526362
   Google Scholar
• Hager, P. 2006. “Nature and Development of Generic Attributes.” In Graduate Attributes, Learning and Employability, edited by P. Hager and S. Holland, 17–47. Dordrecht: Springer.
   Google Scholar
• Hofreiter, T. D., M. C. Monroe, and T. V. Stein. 2007. “Teaching and Evaluating Critical Thinking in an Environmental Context.” Applied Environmental Education & Communication 6 (2): 149–157.10.1080/15330150701598197
   Google Scholar
• Hollweg, K. S., J. R. Taylor, R. W. Bybee, T. J. Marcinkowski, W. C. McBeth, and P. Zoido. 2011. Developing a Framework for Assessing Environmental Literacy. North American Association for Environmental Education. Accessed February 28, 2016. http://www.naaee.net
   Google Scholar
• Hughes, R. 1998. “Considering the Vignette Technique and its Application to a Study of Drug Injecting and HIV Risk and Safer Behaviour.” Sociology of Health and Illness 20 (3): 381–400.10.1111/shil.1998.20.issue-3
   Web of Science ®Google Scholar
• Innovation & Business Skills Australia 2013. “The Vet Practitioner Capability Framework.” Innovation & Business Skills Australia [Online]. Accessed March 6, 2015. https://www.ibsa.org.au/vet-practitioner-capability-framework
   Google Scholar
• Innovation & Business Skills Australia 2015. “About the Vet Practitioner Capability Framework.” Innovation & Business Skills Australia [Online]. Accessed March 6. https://www.ibsa.org.au/about-vet-practitioner-capability-framework
   Google Scholar
• Ison, R. 2010. Systems Practice: How to Act in a Climate-Change World. London: Springer & The Open University.10.1007/978-1-84996-125-7
   Google Scholar
• Kagawa, F. 2007. “Dissonance in Students’ Perceptions of Sustainable Development and Sustainability: Implications for Curriculum Change.” International Journal of Sustainability in Higher Education 8 (3): 317–338. doi: 10.1108/14676370710817174.
   Google Scholar
• Leighton, S. 2010. “Using a Vignette-Based Questionnaire to Explore Adolescents’ Understanding of Mental Health Issues.” Clinical Child Psychology and Psychiatry 15 (2): 231–250.10.1177/1359104509340234
   PubMedGoogle Scholar
• Lenzner, V. V. T. 2011. “A Psycholinguistic Look at Survey Question Design and Response Quality.” PhD, University of Mannheim, Germany [Online]. Accessed February 17, 2015. https://ub-madoc.bib.uni-mannheim.de/29478/1/Lenzner_Dissertation.pdf
   Google Scholar
• Lozano, J. F., A. Boni, J. Peris, and A. Hueso. 2012. “Competencies in Higher Education: A Critical Analysis from the Capabilities Approach.” Journal of Philosophy of Education 46 (1): 132–147.10.1111/jope.2012.46.issue-1
   Web of Science ®Google Scholar
• Martin, S. 2005. “Sustainability, Systems Thinking and Professional Practice.” Systemic Practice and Action Research 18 (2): 163–171.10.1007/s11213-005-4156-7
   Web of Science ®Google Scholar
• Martin, R., B. Maytham, J. Case, and D. Fraser. 2005. “Engineering Graduates’ Perceptions of How Well They Were Prepared for Work in Industry.” European Journal of Engineering Education 30 (2): 167–180.
   Google Scholar
• McNamara, J. 2013. “The Challenge of Assessing Professional Competence in Work Integrated Learning.” Assessment & Evaluation in Higher Education 38 (2): 183–197.10.1080/02602938.2011.618878
   Web of Science ®Google Scholar
• Mochizuki, Y., and Z. Fadeeva. 2010. “Competences for Sustainable Development and Sustainability.” International Journal of Sustainability in Higher Education 11 (4): 391–403.10.1108/14676371011077603
   Web of Science ®Google Scholar
• Mogensena, F., and K. Schnack. 2010. “The Action Competence Approach and the ‘New’ Discourses of Education for Sustainable Development, Competence and Quality Criteria.” Environmental Education Research 16 (1): 59–74.
   Web of Science ®Google Scholar
• Nussbaum, M. 2011. Creating Capabilities: The Human Development Approach. Cambridge, MA: Harvard University Press.
   Google Scholar
• OECD (Organisation for Economic Co-operation and Development) 2012. AHELO: Assessment of Higher Education Learning Outcomes [Online]. Accessed October 30, 2016. http://www.oecd.org/edu/skills-beyond-school/skills-beyond-school/AHELO%20Brochure.pdf
   Google Scholar
• Parker, J. 2010. “Competencies for Interdisciplinarity in Higher Education.” International Journal of Sustainability in Higher Education 11 (4): 325–338.10.1108/14676371011077559
   Web of Science ®Google Scholar
• Parkin, S., A. Johnston, H. Buckland, F. Brookes, and E. White. 2004. “Learning and Skills for Sustainable Development: Developing a Sustainability Literate Society, Guidance for Higher Education Institutions.” In Forum for the Future [Online]. Accessed January 18, 2011. http://www.forumforthefuture.org/files/learningandskills.pdf
   Google Scholar
• Potts, M. K., and C. H. Kleinpeter. 2001. “Distance Education Alumni: How Far Have They Gone?” Journal of Technology in Human Services 18 (3–4): 85–99.10.1300/J017v18n03_06
   Google Scholar
• Presser, S. M., M. P., Couper, J. T. Lessler, E. Martin, J. Martin, J. Rothgeb, and E. Singer. 2004. “Methods for Testing and Evaluating Survey Questionnaires.” Public Opinion Quarterly 68 (1): 109–130.10.1093/poq/nfh008
   Web of Science ®Google Scholar
• Rigby, B. 2010. The Assessment of Graduate Skills: Orientating Students and Standards for an Uncertain Future. Australian Learning and Teaching Council. Accessed October 1, 2016. http://www.graduateskills.edu.au/wp-content/uploads/2010/08/GraduateSkills_ReviewOfAssessment.pdf
   Google Scholar
• RMIT University 2015a. “Graduate Attributes” [Online]. Accessed March 1, 2015. http://www1.rmit.edu.au/teaching/graduateattributes
   Google Scholar
• RMIT University 2015b. “Graduate Attributes: Environmentally Aware and Responsible” [Online]. Accessed March 1, 2015. http://www1.rmit.edu.au/browse;ID=esi2ehsc1kv
   Google Scholar
• Rychen, D., and L. Solganik. 2003. Key Competencies for a Successful Life and a Well-functioning Society. Boston, MA: Hogrefe Publishing Corp.
   Google Scholar
• Sadler, D. R. 2013. “Making Competent Judgements of Competence.” In Modeling and Measuring Competencies in Higher Education: Tasks and Challenges, edited by Weber, S., F. Achtenhagen, and F. Oser, 13–27. Sense [Online]. Accessed December 1, 2016. http://link.springer.com.ezproxy.lib.rmit.edu.au/book/10.1007/978-94-6091-867-4
   Google Scholar
• Sandri, O. 2014. “Good Practice Learning and Teaching for Sustainability in Higher Education.” PhD, RMIT University [Online]. Accessed July 1, 2015. https://researchbank.rmit.edu.au/eserv/rmit:160976/Sandri.pdf
   Google Scholar
• Sandri, O, H. Holdsworth, and I. Thomas. under review. “Assessing Graduate Sustainability Capability Post-degree Completion: Why is it Important and What are the Challenges? International Journal of Sustainability in Higher Education.
   Google Scholar
• Sapsford, R. 1999. Survey Research. London: Sage.
   Google Scholar
• Saris, W. E., and I. N. Gallhofer. 2007. “Concepts-by-Postulation and Cocepts-by-Intuition.” In Design, Evaluation, and Analysis of Questionnaires for Survey Research, 15–29. Wiley [Online]. Available: Accessed February 17, 2015. http://onlinelibrary.wiley.com.ezproxy.lib.rmit.edu.au/book/10.1002/978047016519510.1002/9780470165195
   Google Scholar
• Scott, W., and S. Gough. 2010. “Sustainability, Learning and Capability: Exploring Questions of Balance.” Sustainability 2: 3735–3746.10.3390/su2123735
   Google Scholar
• Segalas, J., D. Ferrer-Balas, M. Svanström, U. Lundqvist, and K. F. Mulder. 2009. “What Has To Be Learnt for Sustainability? A Comparison of Bachelor Engineering Education Competences at Three European Universities.” Sustainability Science 4: 17–27.10.1007/s11625-009-0068-2
   Web of Science ®Google Scholar
• Shephard, K., J. Harraway, B. Lovelock, M. Mirosa, S. Skeaff, L. Slooten, S. T Strack, M. Jowett Furnari, and L., Deaker. 2015. “Seeking Learning Outcomes Appropriate for ‘Education for Sustainable Development’ and for Higher Education.” Assessment & Evaluation in Higher Education 40 (6): 855–866.10.1080/02602938.2015.1009871
   Web of Science ®Google Scholar
• Spiel, C., B. Schober, and R. Reimann. 2013. “Modeling and Measurement of Competencies in Higher Education: The Contribution of Scientific Evaluation.” In Modeling and Measuring Competencies in Higher Education: Tasks and Challenges, edited by S. Weber, F. Achtenhagen, and F. Oser, 195–206. Sense [Online]. Accessed December 17, 2014. http://link.springer.com.ezproxy.lib.rmit.edu.au/book/10.1007/978-94-6091-867-4
   Google Scholar
• Sterling, S. 2009. “Ecological Intelligence: Viewing the World Relationally.” In Handbook of Sustainability Literacy: Skills for a Changing World, edited by A. Stibbe, 77–83. Dartington: Green Books, Foxhole.
   Google Scholar
• Sterling, S., and I. Thomas. 2006. “Education for Sustainability: The Role of Capabilities in Guiding University Curricula.” International Journal of Innovation and Sustainable Development 1 (4): 349–370.10.1504/IJISD.2006.013735
   Google Scholar
• Thomas, I. 2003. “Employers Expectations of Graduates of Environmental Programs: An Australian Experience.” Applied Environmental Education & Communication 2 (1): 49–59.10.1080/15330150301341
   Google Scholar
• Thomas, I., K. Hegarty, and S. Holdsworth. 2012. “The Education for Sustainability Jig-saw Puzzle: Implementation in Universities.” Creative Education 3 (6): 840–846.10.4236/ce.2012.326125
   Google Scholar
• Vargas, P. 2008. Vignette Question. Encyclopedia of Survey Research Methods. Sage Publications Inc. Thousand Oaks, CA: Sage.
   Google Scholar
• van der Velden, R. 2013. “Measuring Competences in Higher Education: What Next?” In Modeling and Measuring Competencies in Higher Education: Tasks and Challenges, edited by S. Weber, F. Achtenhagen, and F. Oser, 207–216. Sense [Online]. Accessed December 5, 2016. http://link.springer.com.ezproxy.lib.rmit.edu.au/book/10.1007/978-94-6091-867-4
   Google Scholar
• Wals, A. E. J. 2010. “Mirroring, Gestaltswitching and Transformative Social Learning: Stepping Stones for Developing Sustainability Competence.” International Journal of Sustainability in Higher Education 11 (4): 380–390.10.1108/14676371011077595
   Web of Science ®Google Scholar
• Wals, A. E. J., and B. Jickling. 2002. “Sustainability in Higher Education: From Doublethink and Newspeak to Critrical Thinking and Meaningful Learning.” International Journal of Sustainability in Higher Education 3 (3): 221–232.10.1108/14676370210434688
   Google Scholar
• Warburton, K. 2003. “Deep Learning and Education for Sustainability.” International Journal of Sustainability in Higher Education 4 (1): 44–56.10.1108/14676370310455332
   Google Scholar
• Wiek, A., L. Withycombe, and C. L. Redman. 2011. “Key Competencies in Sustainability: A Reference Framework for Academic Program Development’.” Sustainability Science 6: 203–218.10.1007/s11625-011-0132-6
   Web of Science ®Google Scholar