The university student experience of face-to-face and online discussions: coherence, reflection and meaning

Abstract

This paper reports on an investigation into learning through discussions by undergraduate social work students. Second-year students studying psychology for social work experienced discussions began with face-to-face tutorials, and then continued for some time after online. This study used closed-ended questionnaires to investigate what students thought they were learning through discussions (their concepts), and how they engaged in the discussions face-to-face and online (their approaches). Significant associations were found among students’ concepts of discussions, approaches and levels of achievement. The results suggest that students who do not understand how discussions can help them to interrogate, reflect on and revise their ideas tended not to approach either face-to-face or online discussions in ways likely to improve their understanding or their levels of achievement. This type of insight is critical for teacher/designers wishing to create university experiences in which discussion is used to promote learning.

Introduction

e-Learning as a key part of the university student learning experience is no longer the preserve of distance teaching universities (HEFCE, 2004; Educause, 2005). It is now common for campus-based universities to invest systematically in e-learning technologies to support the student learning experience. Motivations to use information and communications technology in combination with face-to-face teaching activities include creating new opportunities for learning collaborations and partnerships, strengthening and extending learning communities, promoting new ways of communicating and investigating, and providing better access to an increasingly wide range of discipline-specific educational and research-based pedagogic resources.

While e-learning activities and materials have a lot to offer, it is reasonable to ask how students experience coherence in their learning activity when it is distributed across face-to-face and online contexts. Integrating online learning to complement face-to-face experiences presents students and teachers with some difficulties. In order to embed the online part of the experience, it is sensible and meaningful to use it to help students to attain the intended learning outcomes of the course (Biggs, 2005). But what happens when students are expected to follow a learning activity across face-to-face and online contexts in order to achieve their learning outcomes? We should not assume that following the intent of the teacher/designer throughout a learning activity across face-to-face and online contexts is unproblematic. Certainly we need some evidence about how students experience learning across the combined or ‘blended’ learning contexts, otherwise we may put the outcomes of an increasing amount of the university student learning experience at risk.

This study is designed to investigate a university student experience of learning through discussions that extend across face-to-face and online contexts. The research setting was a course in psychology taken by second-year social work students. As part of their learning activities, students were expected to engage in structured discussions in face-to-face and online tutorial groups. The discussions were intended to help them reflect on, and understand, theoretical underpinnings and key issues in their course. This research study investigates the strength of associations between key aspects of the face-to-face and online student experiences, and the extent to which the students experienced associations between these contexts and their learning outcomes. The main research questions are as follows:

• How do students experience the combination of face-to-face and online discussions?

• Do all students experience them in ways that support their learning?

• What aspects of the student experience of discussions are important for the teacher/designer when considering their systematic use in the student experience?

Theoretical background

The theoretical background for this study includes phenomenographic research on student learning in higher education, and research on reflection as an important aspect of the student learning experience. The former provides the basis for the closed-ended instruments used in the investigation. The latter is used to extend the scope of the research and to offer a way of understanding the motivation of this intervention into the ways the social work students were engaging in discussion.

Research into student learning in higher education

Seminal research on student learning in higher education, over the past 30 years, has systematically identified the relational nature of key aspects of the student experience (Marton & Saljo, 1976a, b; Entwistle & Ramsden, 1983; Ramsden, 2002; Marton & Booth, 1997; Prosser & Trigwell, 1999; Biggs, 2003). These aspects are shown in Figure 1.

Figure 1 The 3P model of learning (after Trigwell & Prosser, 1997)

Studies that underpin Figure 1 have shown that key aspects of the student learning experience in higher education include:

• Students’ conceptions—what they think they are learning: box 5 (Crawford et al., 1998; Prosser & Trigwell, 1999).

• Students’ approaches—what they do when they learn: box 4 (Prosser & Millar, 1989).

• How these aspects are related to their perceptions and the learning context: boxes 3 and 2 (Ramsden, 1991)—and to their prior experiences: box 1 (Prosser & Trigwell, 1999).

A significant part of this research tradition has been the development and use of questionnaires consisting mainly of closed-ended questions to identify and probe these aspects of student learning. The Revised Study Process Questionnaire (Biggs et al., 2001), the Course Experience Questionnaire (Ramsden, 1991) and Conceptions of Learning questionnaires (Crawford et al., 1998) are examples of instruments that have been used to investigate approaches, perceptions and conceptions, respectively. Development of the questionnaires used in the current study has been informed by knowledge of the development processes and structure of these well-established instruments (see Method section).

In earlier research we have reported on the qualitative analysis of interview data in which students have told us about how they learn through discussions (Ellis et al., 2006). The context of that study was similar to the one described in this paper in both structure and content, as it was a pre-requisite for the course investigated in this study. In the previous study we identified qualitatively different conceptions of discussions, and approaches to discussions, in face-to-face and online contexts. These are summarized below.

Categories of conceptions of learning through discussions:

• Conception A: Discussions as a way of challenging ideas and beliefs in order to arrive at a more complete understanding.

• Conception B: Discussions as a way of challenging and improving your ideas.

• Conception C: Discussions as a way of collecting ideas.

• Conception D: Discussions as a way of checking your ideas are right.

The categories of conceptions can be separated into two groups: categories A & B and categories C & D. Categories A & B conceived of discussions as a way of challenging the ideas being discussed to promote holistic understanding. These categories of conceptions were meaningfully related to the course goals and were logically associated with the categories of approaches reported below. In contrast, categories C & D conceived of discussions as a much less interactive way of learning, more about hearing and checking the accuracy of ideas rather than developing and reconceptualizing them.

The previous study also identified qualitatively different approaches to learning through discussions, both face-to-face and online.

Categories of approaches to learning through face-to-face discussions:

• Approach A: Engaging in face-to-face discussions to analyse experiences and opinions through feedback.

• Approach B: Engaging in face-to-face discussions to analyse experiences and opinions.

• Approach C: Engaging in face-to-face discussions to hear other experiences and ideas.

• Approach D: Engaging in face-to-face discussions to fulfil task requirements.

Categories of approaches to learning through online discussions:

• Approach A: Engaging in online discussions to evaluate postings to reflect on key ideas.

• Approach B: Engaging in online discussions to evaluate postings to challenge ideas.

• Approach C: Engaging in online discussions to use postings to add to ideas.

• Approach D: Engaging in online discussions to read postings to avoid repetition.

The categories of approaches A–D in both face-to-face and online discussions are qualitatively different. It is possible to group each set of categories into two. In both sets of categories, those labelled A–B are approaches with an underlying intention of reflection; in face-to-face discussions, the approaches emphasize the analysis of the experiences of others; and in the online discussions, the emphasis is on the evaluation of postings in relation to the ideas being considered. In contrast, categories C–D are qualitatively different. In the face-to-face context, hearing ideas and completing the task requirements were emphasized. In the online context, the use of postings as a way of adding to existing ideas and reading postings to avoiding accidental repetition of others’ postings were emphasized.

Reflection is an important construct underlying the category system presented above. It was prominent in the in-depth interviews with students in the previous study. The students reported reflection as one of the most valuable affordances that the online postings provided. This was true of the ideas that had been expressed in class and the additional ideas that were being posted online. This aspect of the student experience, opportunity for reflection, was one of the main motivations for continuing with the discussion-based teaching strategy in the course under investigation in this study.

Research into reflection as a way of learning

Reflection and reflective practice have been the focus of important research into student learning. Deep reflection is recognized as a higher-order learning activity (Schön, 1983; Biggs, 2003). Reflection on discussions as a way of learning can be usefully conceptualized as reflection occurring during learning and reflection occurring on learning that has taken place (Schön, 1983; Salmon, 2002). Despite superficial differences in terminology, research into online discussions, computer-mediated conferencing and networked learning often focuses on very similar kinds of learning tasks. The results from such studies reveal a degree of convergence: that good online discussions foster effective collaborative learning (Dewiyanti et al., 2005); that students can learn through reflecting deeply on the online submissions made by others if they understand the purpose of the activity and approach it in useful ways (Ellis et al., 2004); that reflection can be part of an active learning structure for online discussions that facilitate the sharing of different viewpoints and ideas to develop understanding (Gilbert & Dabbagh, 2005); and that students do not necessarily have to participate directly in a discussion to benefit from reflecting through listening or reading (McKendree et al., 1998).

The purpose of the current study is to extend previous research to investigate how key parts of the face-to-face and online learning experiences were related, from a student perspective, so that a better understanding, and evidence, of the nature and extent of the success of the intervention could be obtained.

The research site and nature of the educational intervention

Second-year social work students studied psychology as a core part of their undergraduate degree in a metropolitan Australian university. The lecturer embedded structured face-to-face and online discussions into the course, with the aim of helping the students extend their face-to-face discussions into the online environment, thereby providing opportunities for reflection on theoretical ideas and on the practice of social work. The introduction of the blended discussions was accompanied by instruction in class of the value of extending the classroom discussions and their ideas into the online environment. Students were encouraged to use their composition process of writing postings, and the postings of the other students, to reflect on the key ideas of their course.

The students were expected to engage in the discussions to help them to come to terms with knowledge and theories in psychology and to understand their relevance and application to the profession of social work. The online discussions were prompted by the issues and questions that had been raised in the face-to-face tutorials, but often went beyond these. The students were therefore able to extend their face-to-face discussions by making at least two postings (of 200 words each) online during the week following their face-to-face tutorial. They had the choice of starting their own discussion thread or, alternatively, they could reply to a posting made by another student. The students were awarded 13% of their final mark for tutorial participation, determined by the quality of their online postings. All postings by students were made using the discussion tool in WebCT, campus version 4.1.

Participants in this study were recruited by approaching the students in one of their lectures at the end of the 13-week semester. Volunteers were asked to complete three questionnaires. In this cohort of social work students, 48 out of 105 students completed the questionnaires, representing 46% of the cohort.

Research method

Questionnaires

Students who took part in the surveys completed three closed-ended questionnaires: the ‘conceptions of learning through discussions’ questionnaire, the ‘face-to-face approaches to learning’ questionnaire and the ‘online approaches to learning’ questionnaire.

The original development of the questionnaires was informed by previous research (Biggs, 1987; Ramsden, 1991; Crawford et al., 1998; Prosser et al., 2000) and they had been refined in more recent research (Ellis & Calvo, 2004).

The ‘conceptions of learning through discussions’ questionnaire has two subscales; a cohesive subscale and a fragmented subscale. The cohesive subscale investigates a conception of discussions that is orientated towards probing the underlying meaning of topics, their relationship to the learning outcomes of the course and their usefulness in terms of a holistic understanding of the disciplinary issues being discussed. The fragmented subscale investigates a conception of discussions that separates the purpose of the discussions from the context in which they are held. This type of conception is one that emphasizes elements of discussions such as communication skills, finding correct answers or plugging holes in arguments, rather than anything more directly concerned with meaningful understanding. The fragmented and cohesive subscales of the questionnaire are consistent with the theoretical model shown in Figure 1, and their alpha values¹ fell within an acceptable range (α = 0.74, α = 0.89) compared with similar previous studies (0.60 < α < 0.89) (Ramsden et al., 1989; Ellis & Calvo, 2004). Using discussions as a way of reflecting is a constituent part of the coherent subscale: for example, item 11 states ‘Discussing in this subject is like a process of reflection that allows us to better understand the things we study’.

The two questionnaires investigating face-to-face and online approaches had previously been piloted as one questionnaire. Students reported that they preferred to answer items on questionnaires that more clearly separated the two contexts and their answers. This led to the development of a ‘face-to-face approaches to discussions’ questionnaire and an ‘online approaches to discussions’ questionnaire.

The ‘face-to-face approaches to discussions’ questionnaire is divided into two subscales: deep approaches and surface approaches. The deep subscale taps approaches orientated towards understanding ideas from the experience of others, to learn about the ideas of others and to evaluate the way they think about a topic. The surface subscale taps approaches that were less meaning-directed in intent, such as using discussions to complete set tasks, or reduce the need to do as much reading by seeing what others have read. The face-to-face approaches subscales are consistent with the theoretical model shown in Figure 1 and their alpha values (deep α = 0.81, surface α = 0.66) fell within an acceptable range compared with similar previous studies (0.60 < α < 0.89) (Ramsden et al., 1989; Ellis & Calvo, 2004). Reflective evaluation is a constituent part of the face-to-face deep approach subscale. For example, item 11 states ‘I use discussions in class as a way of evaluating the way I am thinking about a topic’.

The ‘online approaches to discussions’ questionnaire is similarly divided into two subscales: deep and surface. The deep subscale taps approaches orientated towards integrating key ideas from postings into reconceived conceptions, both reflecting on ideas before making comment and evaluating the ideas of others using some understanding of who is making the comment. The surface subscale taps aspects such as waiting to the last possible moment before making a posting, worrying about appearing ignorant or posting only because postings are assessed. The online approach subscales are consistent with the theoretical model shown in Figure 1 and their alpha values (deep α = 0.73, surface α= 0.76 fell within an acceptable range compared with similar previous studies (0.60 < α < 0.89) (Ramsden et al., 1989; Ellis & Calvo, 2004). Reflection is a constituent part of a deep approach to online discussions. For example, item five states ‘Online discussions allow me time to reflect on what I wish to say before writing my posting’.

Table 1 presents example items of the subscales and their alpha coefficients.

Table 1. Scales of the questionnaires used in this study, defining items and internal consistency reliabilities

Scale	Number	Defining items
Conceptions of learning through discussions
Cohesive conception (five items, α = 0.89)	3	By discussing I can develop my understanding of the things I am studying in this subject
	11	Discussing in this subject is like a process of reflection that allows us to better understand the things we study
Fragmented conception (three items, α = 0.74)	2	The purpose of discussions in this subject is to learn how to win the argument
	9	Discussing in this subject is just about finding the right answer
Approaches to learning through face-to-face discussions
Face-to-face deep approaches (four items, α = 0.81)	1	I find that at times the discussions in class help me to understand the topic from the experience of others
	5	I find that discussions in class help me to understand the topic being discussed more deeply.
Face-to-face surface approaches (three items, α = 0.66)	6	I discuss issues in class so that I do not have to do as much reading on a topic
	8	I engage in discussions in class only in order to finish a task
Approaches to learning through online discussions
Online deep approaches (four items, α = 0.73)	3	In looking at postings online, I integrate key ideas into my own understanding of a topic
	5	Online discussions allow me time to reflect on what I wish to say before writing my posting
Online surface approaches (three items, α = 0.76)	4	When I look at online postings, my main concern is to avoid posting something that suggests I do not know what I’m talking about
	12	I usually make online postings at the last moment possible

Results

The conceptions and approaches questionnaires were analysed to investigate the extent of coherence across the face-to-face and online contexts in the student experience of discussions. The statistical treatment included correlation, factor and cluster analyses. Correlation analyses investigate relationships between pairs of variables. Factor analyses investigate relationships among groups of variables and add to the integrity to the study. Cluster analyses are used to see whether there are similar groupings of students in the population.

Correlation analyses

Pearson correlation coefficients were used to investigate the strength of the relationships between pairs of variables. Table 2 presents the results of the correlation analyses.

Table 2. Correlations between parts of the student learning experience and achievement

Variable	Cohesive conceptions	Fragmented conceptions	Deep face-to-face approach	Surface face-to-face approach	Deep online approach	Surface online approach	Course mark
Conceptions
Cohesive conceptions	1	−0.20	0.77**	−0.30*	0.41**	0.13	0.10
Fragmented conceptions		1	−0.08	0.35*	0.030	0.089	−0.31*
Face-to-face approaches
Deep face-to-face approach			1	−0.13	0.41**	0.18	−0.10
Surface face-to-face approach				1	−0.10	0.06	−0.09
Online approaches
Deep online approach					1	0.06	0.06
Surface online approach						1	−0.21
Achievement
Course mark (cm)							1

n = 48. *p < 0.01, **p < 0.001, bold = statistically significant.

Table 2 provides some indication of relationships between the face-to-face and online contexts and outcomes. The cohesive conceptions variable shows a large positive association with the deep face-to-face approach variable (r = 0.77, p < 0.00), a negative medium association with the surface face-to-face approach variable (r = −0.30, p < 0.01) and a medium positive association with the deep online approach variable (r = 0.41, p < 0.00). The fragmented conceptions variable shows a medium positive association with the surface face-to-face approach variable (r = 0.35, p < 0.00) and a medium negative association with the course mark (r = −0.31, p < 0.00). While the remaining correlation results are not statistically significant, they are generally in the expected directions.

Factor analyses

Factor analyses look at the structural relationships among groups of variables. Factor analyses group variables together that seem to be related to each other and identify those variables that are not part of each group (see Table 3).

Table 3. Principal components factor analysis of conceptions, approaches and achievement variables

Variable	Factor one	Factor two
Conceptions
Cohesive conceptions	0.89
Fragmented conceptions		0.76
Face-to-face approaches
Deep face-to-face approach	0.89
Surface face-to-face approach		0.56
Online approaches
Deep online approach	0.64
Surface online approach		0.47
Achievement
Course mark		−0.69

Loadings of magnitude less than 0.4 are omitted. Varimax rotation, KMO (Kaiser-Meyer-Olkin) = 0.58. Eigenvalue 2.3 and 1.6, 55% variance explained, n = 48.

In Table 3, factor one shows that a higher score on the cohesive conception variable (0.89) is positively related to the deep face-to-face approach variable (0.89), and the deep online approach variable (0.64). Factor two shows that a higher score on the fragmented conceptions variable (0.76) is positively associated with the surface face-to-face approach variable (0.56), the surface online approach variable (0.49) and negatively related to the course mark (−0.69). The factor analysis suggests that the variable representing a cohesive conception of discussions as a way of learning is strongly associated with a deep face-to-face approach to discussions and a deep online approach to discussions. Likewise the variable representing a fragmented conception of discussions is strongly linked to a surface face-to-face approach to discussions and a surface online approach to discussions, and is negatively related to the course mark.

The factor analysis suggests that the variable representing a cohesive conception of discussions as a way of learning is strongly associated with a deep face-to-face approach to discussions and a deep online approach to discussions. Likewise the variable representing a fragmented conception of discussions is strongly linked to a surface face-to-face approach to discussions and a surface online approach to discussions, and is negatively related to the course mark.

Correlation and factor analyses are used to interrogate data at the level of the variables. To investigate the data at the level of the individual student, a cluster analysis was used. The following methodology draws on a modified version of the approach reported in Prosser et al. (2000).

Hierarchical cluster analysis is a technique that can be used to identify subgroups within a sample on the basis of similarities of the variables being investigated. In this cluster analysis, standardized scores are used for the subscales of conceptions of learning through discussions (fragmented and cohesive), approaches to learning through discussions (deep and surface) and the course mark that all students received. Standardized scores have a mean of zero and a standard deviation of one to make comparisons easier.

The hierarchical cluster analysis using Ward’s technique (Seifert, 1995) revealed two clusters of students, based on the increasing value of the squared Euclidean distance between the clusters (Crawford et al, 1998; Prosser et al., 2000). The results are presented in Table 4.

Table 4. Cluster analysis of parts of the student learning experience and achievement (n = 48)

	Cluster (standardized means)
Variable	Cluster 1 (n = 21)	Cluster 2 (n = 27)	Statistical significance if p < 0.05
Conceptions
Cohesive conceptions	−0.69	0.54	0.001
Fragmented conceptions	0.23	−0.18	–
Face-to-face approaches
Deep face-to-face approach	−0.62	0.48	0.001
Surface face-to-face approach	0.65	−0.50	0.001
On-line approaches
Deep on-line approach	−0.51	0.40	0.001
Surface on-line approach	0.02	−0.02	–
Achievement
Course mark	0.00	−0.01	–

Table 4 shows two groups of students identified in the cluster analysis: one cluster (21 students) scored strongly positive on the fragmented and surface variables, and negatively on the others; and another cluster (27 students) scored strongly positive on the cohesive and deep variables, and negatively on the others.

In Table 4, one cluster of students who held a poor conception of learning through discussions, judged by a large negative score on the cohesive conception variable (−0.69, p < 0.00), had a strong negative score on the deep face-to-face approach variable (−0.62, p < 0.00), a strong positive score on the surface face-to-face approach variable (0.65, p < 0.00) and a strong negative score on the deep online approach variable (−0.51, p < 0.00).

In contrast, the other cluster of students who held a good conception of learning through discussions, judged by a large positive score on the cohesive conception variable (0.54, p < 0.00), had a medium positive score on the deep face-to-face approach variable (0.48, p < 0.00), a large negative score on the surface face-to-face approach variable (−0.50, p < 0.00) and a medium positive score on the deep online approach variable (0.40, p < 0.00).

Discussion and conclusions

The results of the study are promising and consistent with both prior research and the theoretical framework in which the study is conducted. The small size of the sample means that any interpretations of the outcomes should be seen as suggestive of trends and tendencies, rather than as conclusive evidence of relationships. Bearing that in mind, there are a number of implications for thinking about student learning, teaching and further research.

The consistency of the results across the different types of analyses endorses the value of this approach to researching the coherence between face-to-face and online learning contexts. This is an important outcome if we acknowledge that the growth of what is coming to be known as ‘blended learning’ in higher education has resulted in a need to devise new ways of measuring and investigating qualitatively different learning experiences of students when they are expected to follow a learning activity across different learning contexts.

In this study, discussion activities were shown to be fragmented from more meaningful learning when the students lacked an awareness of associations between discussions and the learning outcomes of the course or purpose of the task. The evidence suggesting this is the scales scores for fragmented conceptions and surface approaches in the analyses. If the students did not understand how discussions could help them interrogate, reflect on and revise their ideas, they tended not to approach either the face-to-face or online discussions in ways likely to improve their understanding. This would mean that any reflective aspects of the experience, which were part of the initial motivation for the intervention, would be lost on these students.

It is clear from the analyses that discussion activities stretching across the contexts were coherent for some students. The responses of slightly more than one-half of the students (as measured by the cluster analyses) suggest that the students experienced the discussions in a more coherent and meaningful way. Cohesive conceptions of discussions tended to be strongly associated with deep approaches in face-to-face and online contexts. While there was no strong association with course mark in the cluster analyses, there was association at the level of variables in the factor analyses. This needs closer investigation in further studies. It may be that a larger population sample may find a significant relationship between achievement and the quality of the experience. Overall, the qualitative variation indicated by the analyses within and across the face-to-face and online contexts has important implications for practice.

Much is made of the benefits of online discussion for student learning, including its reflective, collaborative and evaluative affordances (Salmon, 2001; Pilkington & Walker, 2003). The results of this study are consistent with a positive view of the value of online discussions, but suggest a few contingencies that probably need to be considered if discussions are to support deep learning. One is that their value needs to be considered in relation to face-to-face discussions when the student experience contains both: in other words, we cannot simply hope for positive outcomes from online discussions if we insert them in a face-to-face context without considering how their design should intersect with the discussions in the face-to-face context.

Other important contingencies suggested as a way of extracting the most from online discussions is that they are more likely to benefit the student experience of learning if students understand how the discussions are related to the purpose of the task and learning outcomes of their course and if discussions are approached by the students in ways that are likely to help them interrogate, review and reconceive fundamental understandings. These features of good discussions cannot be taken for granted. Without understanding why they are discussing, and how to approach the discussions appropriately, in both face-to-face and o-line contexts, students find it difficult to see the connections between the discussions in class and online and what they are supposed to be learning. For the teacher/designer, this insight is critical. It should be one of the main foundations informing the shape and design of such ‘distributed’ discussions. For some time now, the literature has been telling us that it is unsafe to assume that students know how to discuss online (see, for example, Kaye, 1992; Goodyear, 1996). This study shows that it is unsafe to assume that they will all know how to discuss things effectively face-to-face, despite this activity being around for centuries. When face-to-face and online discussions are both involved, the dangers of some students missing out on learning opportunities are even greater. Further research with larger samples and in other discipline areas would build confidence in these findings. But there are already clear implications that students need to be helped to a better understanding of what they can gain through mindful engagement in productive discussions (whether online, face-to-face or both) and that such an understanding needs to be reflected in the design of discursive learning tasks.

Acknowledgements

The authors are pleased to acknowledge the financial assistance of the Australian Research Council (Grant DP0559282 awarded to P.G., R.A.E. and M.P.) in funding used to carry out this study. They would also like to thank Ana-Maria Bliuc for assistance with the literature search for this study.

Notes

1. Cronbach’s alpha reliability coefficient for internal consistency.