Diversifying methods in educational research: what we learned at Winter School

ABSTRACT

Australasian Association for Engineering Education (AAEE) has been sponsoring a Winter School in Engineering Education Research Methods since 2011. This paper describes how in 2017 attendees at the School applied what they had learned about a little-used data-gathering technique: observation. Starting with a Program Logic analysis, which identifies what an intervention ought to be doing, and hence what kind of evidence needs to be collected to describe its effect, some participants who had attended prior Winter Schools, were given the chance to collect the evidence. They found observation to be much harder to do well than one would think. This paper describes their experience and argues for the use of observational techniques in order to triangulate our data-gathering methods and improve the quality of our educational research. However, we also learned that a great benefit of observation comes from sustained reflection on the process and the data collected. Without such reflection, we argue observation is likely to produce rather thin results.

abbriviations: AAEE - Australasian Association of Engineering Education; JEE - Australasian Journal of Engineering Education; WS2017 - 2017 Winter School

KEYWORDS:

• Qualitative research
• observation
• evaluation

Acknowledgments

The authors would like to thank the organisers, facilitators and participants of the 2017 AAEE Engineering Education Winter School whose contributions and interactions made it possible for data to be collected. Special recognition goes to Keith Willey for his contribution to earlier discussions, and Anne Gardner for her contributions and suggestions throughout the process of writing this paper.

Additional information

Notes on contributors

Esther Matemba

Esther Matemba is a doctoral scholar in the Department of Civil Engineering at Curtin University. Her current scholarly activities are in engineering education with a focus on the development of curriculum for global engineers for Africa, team-based learning, team-teaching and multicultural learning environments.

Lyndal Parker

Lyndal Parker, BSc, is a graduate student at University of Technology Sydney and a casual academic at Sydney University, with research interests in curriculum development, student transition and professional identity development.

Lesley Jolly

Lesley Jolly is an anthropologist who has worked with engineers since 1996 through a variety of research projects, the supervision of PhD students in engineering problems that have social dimensions and through the Australasian Association for Engineering Education. She has run workshops on research methods and educational evaluation and was co-founder of the AAEE Winter School in Engineering Education Research. In 2016, she was inducted into the AAEE Hall of Fame as a distinguished member.