Aligned assessment of technology-mediated field learning

Abstract

Mobile technology-mediated learning is increasingly being adopted in geographical contexts, but considerably less thought has been applied as to how assessment protocols might be adjusted so that the overall learning experience remains appropriately aligned. In this case study, in which a suite of locationally situated mediascapes were constructed and used as part of a 2nd year Human Geography field course to Dublin, we explore our success or otherwise in designing an assessment regime that captures the field experience as a whole. Evaluation, conducted via a focus group, identified that the use of a combined mediascape-essay approach as a major component of the assessment successfully captured the main elements of the learning and teaching experience and facilitated deeper learning.

Introduction

Mobile-technology assisted learning is relatively new within geography and other subjects, although its use is rapidly increasing (Lynch et al., 2008). With this in mind, it is perhaps surprising that discourse regarding the assessment of learning experiences incorporating mobile methods is rare both in geography and beyond. This is a potentially significant omission since Biggs (2003) has highlighted that matching up teaching and learning activities and their assessment is an important component of a constructively aligned programme designed to deepen student learning. The connection between student learning and assessment is well established; students focus their time, by and large, on work that contributes to their grade (Gibbs, 1999).

In looking at connections between mobile-technology assisted learning and assessment within a constructively aligned curriculum, practitioners need to look more closely at the motives and aspirations for incorporating technology. We do not advocate assessing using technology simply because it has been incorporated within the student experience. Rather, we suggest that a significant adjustment in learning and teaching style from traditional norms should trigger reflection as to whether adjustments of assessment are needed, and if so, of what type these should be.

In some cases, mobile technologies offer a more efficient replacement to traditional learning and teaching methods and an opportunity for students to practise basic generic skills. However, they do not otherwise challenge the status quo of the overall student learning experience or outcomes. To alter the pattern of assessment in ways that over-emphasise the role of mobile technologies would be to misalign the programme. Where the aim is to alter and enrich student activities and experiences significantly using novel technology, assessment should ideally also change. This can be problematic since assessing all four learning domains (cognitive, affective, conative, and psychomotor (Reeves, 2006)) is not straightforward and focusing assessment strategies on what is (relatively) easy to measure is both practical, replicable and equitable. This paper explores the assessment of a fieldcourse module to Dublin that seeks to use (inter alia) mobile learning and teaching methods to deepen students’ critical engagement with concepts of national identity, in this case by researching the social construction of ‘Modern Ireland’. Mobile learning and teaching approaches are an integral component of this course.

Methods

Mobile software context: Mediascape

The pedagogic approach we employed draws on the concept of “mediascape” as facilitated within the freely available software mscape. Here, mediascapes were embedded within more traditional field teaching methods involving staff-led tours, discussions and student-led group research projects. A mediascape (m-scape) is composed of sounds, images and video placed in the (urban) landscape which can be activated through the use of a GPS-enabled handheld computer (PDA).

Structured use of mediascape

The multi-modal mediascapes perfomed two roles within the field course. Firstly, in addition to a staff-led introductory tour, they were used to guide students during their initial experiences of the city and to provide training in observation both by example and situated questioning. Secondly, the students themselves developed mediascapes as a digital format for expression and critical reflection on their findings in the field. In small groups, students collected photographic and sound-based materials to expose critical processes behind the material rebuilding of Dublin, while others compiled their observations regarding the cultural representations of Irishness in the city. In this, we aimed to align our instructional design with the module goals and content, and also to align learner tasks with those we had exemplified as instructors when constructing the mscapes. Our approach links with contemporary research identifying how photographs can be “active players in the construction of a range of different kinds of geographical knowledge” (Rose, 2008), and seeks to encourage personal reflection by our students through active learning (Healey, 2005).

The fieldcourse was assessed using a group presentation, an individual, traditional field notebook and a student-led project report consisting of a group mediascape and supporting individual reflective essay.

Evaluation

An ongoing programme of evaluation is being carried out throughout the development of the mobile approaches and strategies in the Department of Geography at Leicester, in which the consideration of aligned assessment to cement learning is one theme.

In this paper, data from the reflective diaries of students, a focus group, plus staff reflections on the style and content of the group mediascapes, are used to form a view on the novel assessment approaches we adopted. In particular, we investigate the degree to which the assessment regime was able, through technological affordances, to convey cognitive, affective, conative and psychomotor aspects of the students’ experiences.

Findings

Initial results at the end of the assignment period for the field trip course demonstrated a variety of advantages of incorporating ICT within the assessment process in addition to the initial field learning. We focus specifically on assessment-related aspects in this paper.

Firstly, students were clear that the mscape assignment allowed a fuller expression of their ideas than text alone. The reflective essay and mscape were considered integral to each other by both staff and students alike; together they allowed considerably more information content to be presented. Further, as the student quotes below indicate, the materials incorporated clearly show both cognitive and affective aspects of learning:

“I definitely think the Mscape was better than doing a whole essay on like the area of the Monto, mainly because in the Mscape you could put across more emotional things … like you could portray things that you could never be able to write, like a picture speaks a thousand words … It was just a lot more emotive and just easier to like show what you felt in the area in an Mscape.”

“To support the amount of information we had in the Mscape in an essay, it would be epically long.”

“Being able to add in music and like other people talking, the tour guide, that added like an extra dimension, it’s a different way of putting things across that is much better than just reading an essay.”

Linking with this freedom to communicate, and to incorporate personal experiences, some students felt liberated at being able to communicate in less formal ways. The context surrounding the quotation below provides for an interpretation in which students appreciate freedom from the pincers of grammatical correctness and are able to include, for example, an “epic track” within a formal assessment; they can dare to play with ideas in new ways.

“It’s less structured as well, like you feel more liberated doing it … Like in an essay you have to worry about how someone’s going to read it and things like that, whereas here you can just literally put in what you want to put in and not worry.”

This issue of structure can be seen from two perspectives. In their review on curriculum design principles, Meyers & Nulty (2009) summarise material that suggests students adopting a deep learning approach form highly structured knowledge. The mscape approach clearly does not foster this. However, it does have other advantages that reflect deeper learning consistent with a more aligned module, for example the fostering of thinking that allows more elaborate non-linear connectivity in knowledge structures to be explored. To echo Hartnell-Young and Vetere (2008), “While we and the teachers saw value in linear narratives, which we are used to, it is a sobering thought to consider what we, as researchers, missed about learning in the rich content we saw, and what teachers might be missing every day”. The format also allowed the free-thinking venturing of ideas and expression of theories, again an expression of deeper learning and a well-aligned module. We had not forseen, for example, that students would pose questions back to us as part of their digital mscape materials.

Collating content for the mediascape, involving the making and expressing of connections between movement in the field and mental processes, appealed to a wide range of students. Aspects of technical construction, involving psychomotor skills at a different level, had a more limited appeal. Even then, within a group context, students still preferred the format to an assignment assessed solely by traditional essay; the approach offered those with different conative styles spaces within which to act, partly owing to the range of sensory experience afforded by an m-scape both during its construction and use.

“I quite enjoyed doing our Mscape at home, but I think that’s because I am quite sort of like technically minded and I sort of enjoyed seeing how it worked. Whereas I know other people in our group weren’t that fussed at all about how it worked or what. They enjoyed putting the stuff into it, but the actual construction of it they wouldn’t care at all really, and they still wouldn’t be able to know what went into it.”

“Like me, I wouldn’t be able to construct one I don’t think without the help of a lot of other people in the group, but then I’d still prefer to do an Mscape as opposed to write an essay. So even though I might not be able to make a great one, I’d still prefer yeah, I’d still prefer to do it.”

“Beats doing an essay any day to be honest.”

Conclusions

In this case study, students themselves used mobile devices to participate in the building of further layers of evidence on research themes related, for example, to the material rebuilding of Dublin’s built environment (issues of urban regeneration and gentrification), cultural representations of Irishness in the city and connections to key nodal points within the Irish diaspora. While encouraging students to visualize and synthesize intellectually challenging theoretical concepts about the relationality of urban space (Amin and Thrift, 2002) in new, easily accessible ways, the work also introduced human geography undergraduates to new generation digital technologies.

Implicitly, in the process of building their m-scapes, students recorded their own learning; pedagogically, the approach therefore not only trained students in a leading edge research methodology but provided a means of recording reflective accounts of their fieldwork experiences. That is, we deliberately echoed the mode of the student learning experience, and the many aspects of learning and meaning constructed by the students’ activities, within the assessment process. The student feedback firmly suggests that elements of deeper learning, a sign of a well-aligned module, were fostered in this novel assessment approach. To date little has been written about assessment and alignment in the context of mobile learning, either in Geography or beyond. These early results suggest that further work exploring how we align modules in the context of mobile learning deserves further exploration.

Acknowledgements

This work was financially supported by the University of Leicester New Teaching Initiative Fund. Our appreciation also goes to the students who contributed their opinions via a focus group.